-- Logs begin at Wed 2022-08-10 10:21:44 UTC, end at Wed 2022-08-10 11:51:03 UTC. -- Aug 10 11:02:19.650615 dv-cinder-850618-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 10 11:02:19.659653 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 10 11:02:19 2022] *** Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: nodename: dv-cinder-850618-2-iscsi Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: machine: x86_64 Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: clock source: unix Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: pcre jit disabled Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: detected number of CPU cores: 4 Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: current working directory: / Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: detected binary path: /usr/bin/uwsgi-core Aug 10 11:02:19.661665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: your processes number limit is 47680 Aug 10 11:02:19.662270 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: your memory page size is 4096 bytes Aug 10 11:02:19.662270 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: detected max file descriptor number: 2048 Aug 10 11:02:19.662270 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: lock engine: pthread robust mutexes Aug 10 11:02:19.662270 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: thunder lock: enabled Aug 10 11:02:19.662270 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 10 11:02:19.662270 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 10 11:02:19.702304 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: Python main interpreter initialized at 0x55990f0c99c0 Aug 10 11:02:19.702304 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: python threads support enabled Aug 10 11:02:19.702304 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: your server socket listen backlog is limited to 100 connections Aug 10 11:02:19.702304 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: your mercy for graceful operations on workers is 90 seconds Aug 10 11:02:19.702665 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: mapped 403077 bytes (393 KB) for 2 cores Aug 10 11:02:19.702742 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: *** Operational MODE: preforking *** Aug 10 11:02:19.702983 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: *** uWSGI is running in multiple interpreter mode *** Aug 10 11:02:19.702983 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: spawned uWSGI master process (pid: 80615) Aug 10 11:02:19.703186 dv-cinder-850618-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 10 11:02:19.704626 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: spawned uWSGI worker 1 (pid: 80628, cores: 1) Aug 10 11:02:19.704626 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: spawned uWSGI worker 2 (pid: 80629, cores: 1) Aug 10 11:02:19.705750 dv-cinder-850618-2-iscsi devstack@keystone.service[80615]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 10 11:02:25.236490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 11:02:25.236490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: from cryptography.utils import int_from_bytes Aug 10 11:02:25.275563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 11:02:25.275563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: from cryptography.utils import int_from_bytes Aug 10 11:02:25.275708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 11:02:25.275708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: from cryptography.utils import int_from_bytes Aug 10 11:02:25.275708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 10 11:02:25.275708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: from cryptography.utils import int_from_bytes Aug 10 11:02:25.287174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 11:02:25.287372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 11:02:25.287565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] command line args: [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 11:02:25.287834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 11:02:25.288034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ================================================================================ {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 11:02:25.288344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] admin_token = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.288656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] config_dir = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.288891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.289095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] config_source = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.289308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] debug = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.289529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.290713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.290713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.290943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 10 11:02:25.291069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.291192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 10 11:02:25.291192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] command line args: [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 10 11:02:25.291398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.291520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 10 11:02:25.291520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ================================================================================ {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 10 11:02:25.291715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.291715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_config_append = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.291852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] admin_token = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.291932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] config_dir = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_dir = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] config_source = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_file = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] debug = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.292932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_config_append = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.293836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_dir = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_file = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.294998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.295844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.296936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] publish_errors = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.297944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] publish_errors = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] standard_threads = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.298987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] standard_threads = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] use_journal = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] use_json = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.299972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] use_stderr = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] use_journal = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] use_syslog = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] use_json = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] use_stderr = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] use_syslog = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.300930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.301032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 10 11:02:25.301236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.301319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.301471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.301601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.301727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.301924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.302977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.303145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.303387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.303732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.303894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.304883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.305826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.306977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.307955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308110 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.308888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.309956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.310978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.311212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.311367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.311500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.311638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.311767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.311902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.312881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.313935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.314975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.315989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.316980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.317975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.318970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.319919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.320956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.321108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.321705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.321966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.322935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.323876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324765 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.324998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325229 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.325934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.external = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.326966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.external = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.password = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] auth.token = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.327931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.password = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] auth.token = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.328893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.329979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.330941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.331922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.332945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.333955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.334984 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.335947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.336896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.337574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.337746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.337826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.337993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.338939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.339972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.340937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.341912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.342991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343224 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.343941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.344998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.345955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.346881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.347984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.348946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.349957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.350905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.351982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.352943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.353329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.353585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.353664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.353834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.353915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.354889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.355979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.356963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.357970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.358905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.359871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.360846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] role.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] role.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] role.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.361941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] role.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.362978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.363948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.364916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.365968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.366957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.367884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.368093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.368170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.368434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.368519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.369911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.370856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.371826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.372947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.373977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.374124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.374209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.374364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.374445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.374681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.connection = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.374931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.connection = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.375987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.376966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.377935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.378882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.379034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.379113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.379194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 11:02:25.379268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.379497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 10 11:02:25.379665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 10 11:02:25.409619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.409836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.410065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.410610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.410843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.411068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.419902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.423205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.448823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.449052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.451850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.452106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.501703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.502220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.551976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.552231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.553171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.553662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.616619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.616850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.617964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.618741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.641801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.641995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.642943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.643075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.643206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.643337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.643466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.643604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.654098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.654388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.666768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.666997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.667222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.667370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.667499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.667625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.667750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.667875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.668106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.668234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.668377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.668508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.668508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.668508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 10 11:02:25.685606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.685815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 11:02:25.686153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.686306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.686424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 11:02:25.686527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 11:02:25.686641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.686842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.686925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.687045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 10 11:02:25.687136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.687321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.687401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.687586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.687847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.688133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.700929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.707683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.707771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.710729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.710813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.711017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.711099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.711301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.711382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.711580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.711662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.711884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.712943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.713051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.713134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.713329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.713411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.713603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.713689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.736362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.737046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.737843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.738254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.738522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.738803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.743579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.744096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.744187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.744262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.744348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.744472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.750842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.750968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.751131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.751254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.755270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.755365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.755528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.755609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.755822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.755913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756224 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.756974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.757053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.757239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.757317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.773679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.773776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.773962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.774044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.777933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.778120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.781076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.781173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.781370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.781452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.785744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.785832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.788898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.788982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.792074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.792188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.792455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.792653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.792735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.792933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.793016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.793209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.793292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.793495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.793577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.810117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.810393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.810478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.810659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.810755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.810936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.811019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.811200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.817799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.818115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.819332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.819609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.819690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.819953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.822997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.823281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.823504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.823587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.823792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.823874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.824115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.824202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.824420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.824702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.835394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.835675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.835957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.836098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.836386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.836498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.836670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.836782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.837005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.837247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.837574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.837673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.837758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.838008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.854385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.854485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.857334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.857439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.860307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.860597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.861885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.862181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.863354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.863637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.863717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.863925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.864033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.864299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.871074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.871331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.872700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.872964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.875596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.875884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.875982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.876196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.876292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.876475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.876567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.876752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.885120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.885412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.885514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.885691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.885787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.886035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.892525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.892811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.892917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 10 11:02:25.893091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.893184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.893366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.893460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.893636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.893729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.893907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.894836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.895916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.896054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.896201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.896297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.896481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 10 11:02:25.930184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55990f0c99c0 pid: 80629 (default app) Aug 10 11:02:25.930426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55990f0c99c0 pid: 80628 (default app) Aug 10 11:02:25.934068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8b7fa-64ef-4e2f-a242-9384f5b4bc89 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:25.934349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8b7fa-64ef-4e2f-a242-9384f5b4bc89 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:25.934618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8b7fa-64ef-4e2f-a242-9384f5b4bc89 None None] PATH_INFO: `/v3` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:25.939125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Wed Aug 10 11:02:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:34.551876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:34.552472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:34.552472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:34.557395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:02:34.562392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:34.562776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:34.562931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:34.633198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: INFO dbcounter [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] Registered counter for database keystone Aug 10 11:02:34.654402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG oslo_db.sqlalchemy.engines [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 11:02:34.660539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writer thread running {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 11:02:34.835627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG passlib.handlers.bcrypt [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80629) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 11:02:34.836068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG passlib.handlers.bcrypt [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80629) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 11:02:34.892033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:34.893556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:35.021036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 11:02:35.155499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:35.225309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: INFO dbcounter [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] Registered counter for database keystone Aug 10 11:02:35.244943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG oslo_db.sqlalchemy.engines [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 10 11:02:35.251293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writer thread running {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 10 11:02:35.371300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e38a5-1222-4779-9bde-9dc340fca7c7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:35.375631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:35.376281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:35.376562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:35.376848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:35.385905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:35.438507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 11:02:35.438785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d2adb-dc1b-4671-99d1-3b71418009a6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:35.450117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Wed Aug 10 11:02:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:02:35.456142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:35.470660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23748679-7a1f-4a88-9283-0fde7bc7804f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:35.475077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:35.476207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:35.476487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:35.476758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] PATH_INFO: `/v3/projects/1f5409bf396e4a7cbe05519266660a63` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:35.478248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] RBAC: Authorizing `identity:get_project(project_id=1f5409bf396e4a7cbe05519266660a63)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:35.529660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING keystone.common.rbac_enforcer.enforcer [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 10 11:02:35.530005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655026b6-8daa-469f-8b6e-acccc810103b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:35.534735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Wed Aug 10 11:02:35 2022] GET /identity/v3/projects/1f5409bf396e4a7cbe05519266660a63 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:37.524683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6284f-be02-4501-97c5-720c147fd36c None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.525220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6284f-be02-4501-97c5-720c147fd36c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.525220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6284f-be02-4501-97c5-720c147fd36c None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.529288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:37.534591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.535074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.535171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.556993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:37.558495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:37.566180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 10 11:02:37.588901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6284f-be02-4501-97c5-720c147fd36c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:37.600593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6284f-be02-4501-97c5-720c147fd36c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:37.605666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462139f-fef6-4cf5-a1cc-0c7c0c433975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:37.606251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462139f-fef6-4cf5-a1cc-0c7c0c433975 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.606526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462139f-fef6-4cf5-a1cc-0c7c0c433975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.606794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462139f-fef6-4cf5-a1cc-0c7c0c433975 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.614200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462139f-fef6-4cf5-a1cc-0c7c0c433975 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:37.615924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462139f-fef6-4cf5-a1cc-0c7c0c433975 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:37.624819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Wed Aug 10 11:02:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:02:37.629873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:37.638950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fddb0e-838d-4bf0-b8bf-8742ef99064b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:37.643194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d26e79-2273-40d9-865d-8d0a55f8ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:37.643803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d26e79-2273-40d9-865d-8d0a55f8ebad admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:37.644116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d26e79-2273-40d9-865d-8d0a55f8ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:37.644387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d26e79-2273-40d9-865d-8d0a55f8ebad admin admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:37.645705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d26e79-2273-40d9-865d-8d0a55f8ebad admin admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:37.647487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d26e79-2273-40d9-865d-8d0a55f8ebad admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:37.649008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Wed Aug 10 11:02:37 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:39.718922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b760160-268d-4c1f-aa18-1a8dba39ad00 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.719406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b760160-268d-4c1f-aa18-1a8dba39ad00 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.719406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b760160-268d-4c1f-aa18-1a8dba39ad00 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.720734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:39.726020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.726447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.726580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.745187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:39.746392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:39.784238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da207d-ece9-4830-b404-d48f8b398ddf None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.784522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da207d-ece9-4830-b404-d48f8b398ddf None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.784795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da207d-ece9-4830-b404-d48f8b398ddf None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.786090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:02:39.792199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.792484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.792755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:39.827289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:39.835353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG passlib.handlers.bcrypt [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80628) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 10 11:02:39.835776 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG passlib.handlers.bcrypt [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80628) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 10 11:02:39.850585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:39.874577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:39.881977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:39.883670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:39.954345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed266b9-6b4b-4451-84e3-7adb892e8a4c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:39.983612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:02:39.991235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:39.991878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:39.992192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:39.992469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.003665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:40.004895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:40.012676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:40.013735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:40.014829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:02:40.019789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.099223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.103505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b8420-88a6-4e4c-8db7-6ff6beb71225 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.106206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.117709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f2299e-8aed-44d2-b196-ba3c43e59bda None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.140814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.141672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.143448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.143448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.154364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.154966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.155243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.155512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.156195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:40.156497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.157872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.169345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:40.174922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.214051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.215610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.319717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 8/16] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:02:40.325980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.328836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.380889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb4b75-d6c6-4cc2-969c-cf459b65d18e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.416813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.417446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.417722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.417995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.419114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.420605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.422154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:40.425440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69460e-972f-4d24-a5d5-7198ccae1075 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.427414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.441734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.459191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.460076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.460358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.460635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.479595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.481098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:02:40.499783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:40.500852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:02:40.501814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:02:40.501814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:02:40.501814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:02:40.501814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:02:40.506719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.532969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f5812-9aba-4d28-bec9-2e87d6bf8e60 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.563353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.568832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.569465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.569791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.570010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.571672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.573158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.574702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:40.580307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:40.612380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73253e-56e2-4b37-928b-b5cfa7a2f8c1 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.646689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.647293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.647572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.647842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.648516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:40.648817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.650320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37873de4-e856-4aa4-b9c5-e789a84bd2cc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.660200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:40.667353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:40.680865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19d213-6ad6-4abc-8653-7bd2620133e8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:40.718252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:40.718888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:40.719168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:40.719442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:40.720208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.domain.id=default, user.id=0b3cf49f311d4c78b75ab2d7b48164db) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:40.720515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:40.722047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8ece2-d59d-4afb-ae2b-4c583fd6d76d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:40.730104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1322 bytes} [Wed Aug 10 11:02:40 2022] GET /identity/v3/role_assignments?user.id=0b3cf49f311d4c78b75ab2d7b48164db&scope.domain.id=default&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:43.196128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7277eda-8059-4f86-bd1a-94a28f554fdb None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.197127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7277eda-8059-4f86-bd1a-94a28f554fdb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.197626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7277eda-8059-4f86-bd1a-94a28f554fdb None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.199161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:43.216531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a411-bb72-4c80-95b6-05d2627cf750 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.217012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a411-bb72-4c80-95b6-05d2627cf750 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.217088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792a411-bb72-4c80-95b6-05d2627cf750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.245031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f792a411-bb72-4c80-95b6-05d2627cf750 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:43.246255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f792a411-bb72-4c80-95b6-05d2627cf750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:43.325998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:43.360603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7277eda-8059-4f86-bd1a-94a28f554fdb None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:43.434976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e953-7fa3-497e-b325-cf4a7c79d810 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.435256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e953-7fa3-497e-b325-cf4a7c79d810 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.436237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1e953-7fa3-497e-b325-cf4a7c79d810 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.438166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:43.450048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62087-f559-4af8-a788-e43e24031fb1 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.450601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62087-f559-4af8-a788-e43e24031fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.451552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf62087-f559-4af8-a788-e43e24031fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.490098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-edf62087-f559-4af8-a788-e43e24031fb1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:43.491331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-edf62087-f559-4af8-a788-e43e24031fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:43.556850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7277eda-8059-4f86-bd1a-94a28f554fdb None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:43.599308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:43.638077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:43.638721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.639000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.639274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.660397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 14/25] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:02:43.662120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28ea6e-f514-4af9-8b32-b094ff25298d None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.662406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28ea6e-f514-4af9-8b32-b094ff25298d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.662678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28ea6e-f514-4af9-8b32-b094ff25298d None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.664488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:43.676391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 15/26] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:43 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:43.677671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.677955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.678228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:43.682963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:43.683809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:02:43.685218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 12/27] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:43.691415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:43.696799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:43.717702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:43.718828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:43.812122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 16/28] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:43.813192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:43.935234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339b6a8-9d70-422a-86e1-b321c63b88b4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:43.940568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74d962-d0fd-45a1-8df5-045256dd7e72 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:43.970964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:43.972061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.972847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.973475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.974453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:43.974954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:43.976563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e5497-dd00-4444-b2ff-7d4041dd4d09 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:43.978101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:43.978734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.979011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.979284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.981441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:43.990675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:43.999266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:43.999379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 13/29] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:43.999379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d01e3-f7aa-4c55-b31d-8e9d8de55aa2 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:43.999379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d01e3-f7aa-4c55-b31d-8e9d8de55aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:43.999379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d01e3-f7aa-4c55-b31d-8e9d8de55aa2 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:43.999379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 14/30] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:43.999379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d01e3-f7aa-4c55-b31d-8e9d8de55aa2 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:02:44.015232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:44.016175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:02:44.016963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:02:44.016963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:02:44.016963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:44.016963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 17/31] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:44.018018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.048876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:44.114225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d01e3-f7aa-4c55-b31d-8e9d8de55aa2 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.132758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557b4214-d1b8-4e92-b5ba-ae7a7452a94a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.150949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:44.151616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.151896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.152191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.152850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:44.154206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:44.170555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ac0de-02e1-42bf-981a-c1ed66acf572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:44.171245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ac0de-02e1-42bf-981a-c1ed66acf572 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.171593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ac0de-02e1-42bf-981a-c1ed66acf572 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.171784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ac0de-02e1-42bf-981a-c1ed66acf572 None admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.173228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ac0de-02e1-42bf-981a-c1ed66acf572 None admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:44.174678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ac0de-02e1-42bf-981a-c1ed66acf572 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:44.176596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 18/32] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:44 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:44.182717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.183003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.183272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.201778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:44.202960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:44.255078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:44.280018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 19/33] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:44.280018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.401642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e5163-6c89-41d9-88ea-b1435dc3f8ff None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.435009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 15/34] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 10 11:02:43 2022] POST /identity/v3/roles => generated 219 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:44.437266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.437762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:44.438383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.438660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.438955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.439686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:44.439987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:44.441451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:44.461396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 20/35] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:44.463027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.518045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:44.562524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:44.567645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1388ff-4c5a-491e-a80d-2305e742aa1e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.581263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a24eb6d-6a4a-4e72-a409-bcefb4eb1f71 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.608958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:44.609615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.609895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.610166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.611910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:44.613944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:44.616736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 16/36] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:44.622235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.625517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078e640-de0f-4abe-a344-efb2216e028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:44.626151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e640-de0f-4abe-a344-efb2216e028a None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.626429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e640-de0f-4abe-a344-efb2216e028a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.626707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078e640-de0f-4abe-a344-efb2216e028a None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.627765 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078e640-de0f-4abe-a344-efb2216e028a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:44.629466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078e640-de0f-4abe-a344-efb2216e028a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:44.723756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d1f16-eade-4923-a8fd-44f26fa9b7fc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.757427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d0c66c-2142-4e38-80fb-fd8e4e298468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:44.758035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0c66c-2142-4e38-80fb-fd8e4e298468 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.758824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0c66c-2142-4e38-80fb-fd8e4e298468 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.759591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d0c66c-2142-4e38-80fb-fd8e4e298468 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.760270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0c66c-2142-4e38-80fb-fd8e4e298468 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:44.761693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d0c66c-2142-4e38-80fb-fd8e4e298468 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:44.835496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 21/37] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 10 11:02:44 2022] POST /identity/v3/roles => generated 225 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:44.837630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11953bad-c776-43c8-86ed-c209faea0ab7 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.837924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11953bad-c776-43c8-86ed-c209faea0ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.838201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11953bad-c776-43c8-86ed-c209faea0ab7 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.839417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 22/38] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:44.840510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e605e-226b-4824-8724-b4316533078c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.840794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e605e-226b-4824-8724-b4316533078c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.841068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e605e-226b-4824-8724-b4316533078c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.844065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 23/39] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:44.844065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e605e-226b-4824-8724-b4316533078c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:44.897459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 17/40] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:02:44 2022] POST /identity/v3/projects => generated 301 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:44.898833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a65bf-82dc-4074-890c-d7d443af49bb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:44.899713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a65bf-82dc-4074-890c-d7d443af49bb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:44.900208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a65bf-82dc-4074-890c-d7d443af49bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:44.931246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-8e9a65bf-82dc-4074-890c-d7d443af49bb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:44.933023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8e9a65bf-82dc-4074-890c-d7d443af49bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:44.948897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:44.989692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e605e-226b-4824-8724-b4316533078c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:44.995376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:45.028024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 18/41] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:45.028024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:45.028024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:45.028024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:45.045072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:45.045858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:45.046077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:45.046181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:45.047294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:45.047409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] PATH_INFO: `/v3/domains/default/users/0b3cf49f311d4c78b75ab2d7b48164db/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:45.049766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0b3cf49f311d4c78b75ab2d7b48164db, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:45.055710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:45.126403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 19/42] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:45.127665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:45.177928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 24/43] 10.222.0.13 () {58 vars in 1361 bytes} [Wed Aug 10 11:02:44 2022] PUT /identity/v3/domains/default/users/0b3cf49f311d4c78b75ab2d7b48164db/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:02:45.179017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:45.221238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:45.247102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b4f0a-edbc-4f4f-a2a9-67dfc4fef2c5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:45.285195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e4037-2955-49b3-9aad-ec3cc83e13d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:45.285818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e4037-2955-49b3-9aad-ec3cc83e13d3 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:45.286126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e4037-2955-49b3-9aad-ec3cc83e13d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:45.286395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e4037-2955-49b3-9aad-ec3cc83e13d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:45.287065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e4037-2955-49b3-9aad-ec3cc83e13d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:45.288516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e4037-2955-49b3-9aad-ec3cc83e13d3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:45.292950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731175d-cdb7-4ce0-a4b8-1ca6246a7240 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:45.328872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:45.329425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:45.329706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:45.329981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:45.331612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:45.333349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:45.334737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 25/44] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:45.341085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:45.373475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:45.396299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 20/45] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 10 11:02:45 2022] POST /identity/v3/roles => generated 223 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:45.447120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16feca-b28a-4790-971f-ab8ab3af0884 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:45.484914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb5fa9-0b4f-40a8-b549-311deb89cbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:45.485539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb5fa9-0b4f-40a8-b549-311deb89cbe2 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:45.485821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb5fa9-0b4f-40a8-b549-311deb89cbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:45.486092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb5fa9-0b4f-40a8-b549-311deb89cbe2 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:45.486751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb5fa9-0b4f-40a8-b549-311deb89cbe2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:45.488262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb5fa9-0b4f-40a8-b549-311deb89cbe2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:45.581127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 26/46] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:02:45 2022] POST /identity/v3/projects => generated 312 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:02:47.206681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3d22-88c8-4b7b-a610-6e7b69fbc02a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.207196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3d22-88c8-4b7b-a610-6e7b69fbc02a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.207280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c3d22-88c8-4b7b-a610-6e7b69fbc02a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.208611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 21/47] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:47.214422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854558a6-2959-47da-a671-b43404b5733b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.214849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854558a6-2959-47da-a671-b43404b5733b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.214972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854558a6-2959-47da-a671-b43404b5733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.232934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-854558a6-2959-47da-a671-b43404b5733b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:47.234441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-854558a6-2959-47da-a671-b43404b5733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:47.299361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 27/48] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:47.322885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c3d22-88c8-4b7b-a610-6e7b69fbc02a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.423629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c3d22-88c8-4b7b-a610-6e7b69fbc02a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:47.457218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf058d31-3375-411b-a4c8-12d9e6fe7ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:47.457770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf058d31-3375-411b-a4c8-12d9e6fe7ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.458050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf058d31-3375-411b-a4c8-12d9e6fe7ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.458321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf058d31-3375-411b-a4c8-12d9e6fe7ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.464916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:47.468671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf058d31-3375-411b-a4c8-12d9e6fe7ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-cf058d31-3375-411b-a4c8-12d9e6fe7ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:47.473382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:47.476257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:02:47.477002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 22/49] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:47.480677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854558a6-2959-47da-a671-b43404b5733b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.496387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:47.578793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854558a6-2959-47da-a671-b43404b5733b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:47.582514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31dd07d-58c4-42b6-b8a7-86890df3a8ae None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.582800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31dd07d-58c4-42b6-b8a7-86890df3a8ae None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.583074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31dd07d-58c4-42b6-b8a7-86890df3a8ae None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.584510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 23/50] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:02:47.589431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.589709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.589982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.607241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:47.608464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:47.612922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:47.613535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.613819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.614114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.614767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:47.615073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:47.616494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:47.624963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 28/51] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:47.630364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.677162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 24/52] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:02:47.700488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.707039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:47.730248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def4040-d048-4877-9012-71cd7f5b3763 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:47.763025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:47.763650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.763932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.764237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.765378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:47.766829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:47.768424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 29/53] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:47 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:47.773745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.799814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2690d-6503-485e-afc9-d0109e82b12c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:47.825957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:47.833470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:47.836039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.836039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.836039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.836039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:47.837839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:47.839400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 25/54] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:47.845106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.876980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1a067f-dfa5-4e57-97d9-f0a89d4ac0fb None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:47.910457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:47.911082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.911359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.911628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.912780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:47.914208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:47.916136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 30/55] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:47.921595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:47.935929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:47.942911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cffd1-693e-4d15-81ed-2187531d7d5f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:47.976282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d524a8-3875-4ea4-b9ff-34f1f44ec5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:47.976891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d524a8-3875-4ea4-b9ff-34f1f44ec5b1 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:47.977170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d524a8-3875-4ea4-b9ff-34f1f44ec5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:47.977441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d524a8-3875-4ea4-b9ff-34f1f44ec5b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:47.978075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d524a8-3875-4ea4-b9ff-34f1f44ec5b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:47.979520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d524a8-3875-4ea4-b9ff-34f1f44ec5b1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:48.023591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923548d-f0ce-4f6a-bbd5-343a57eca8a5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:48.050342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:48.050935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:48.051212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:48.051481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:48.052285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.domain.id=default, user.id=0b3cf49f311d4c78b75ab2d7b48164db) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:48.052590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:48.054273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba90b4e-8cc0-4585-8f60-c7b5682abb00 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:48.060275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 31/56] 10.222.0.13 () {56 vars in 1322 bytes} [Wed Aug 10 11:02:47 2022] GET /identity/v3/role_assignments?user.id=0b3cf49f311d4c78b75ab2d7b48164db&scope.domain.id=default&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 530 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:48.095894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 26/57] 10.222.0.13 () {60 vars in 1140 bytes} [Wed Aug 10 11:02:47 2022] POST /identity/v3/projects => generated 298 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:02:50.145056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8cd7-94ee-43b5-9370-8acd5cc3cafd None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:50.145571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8cd7-94ee-43b5-9370-8acd5cc3cafd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:50.145653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab8cd7-94ee-43b5-9370-8acd5cc3cafd None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:50.146928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 32/58] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:50.152273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:50.152666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:50.152826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:50.170861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:50.172114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:50.214299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:50.236393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 27/59] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:50.260596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8cd7-94ee-43b5-9370-8acd5cc3cafd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:50.335683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:50.359395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab8cd7-94ee-43b5-9370-8acd5cc3cafd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:50.393112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d702f04-e85b-45ec-a8a6-32bd68022c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:50.393731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d702f04-e85b-45ec-a8a6-32bd68022c9b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:50.394010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d702f04-e85b-45ec-a8a6-32bd68022c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:50.394279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d702f04-e85b-45ec-a8a6-32bd68022c9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:50.395424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d702f04-e85b-45ec-a8a6-32bd68022c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:50.396855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d702f04-e85b-45ec-a8a6-32bd68022c9b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:50.398396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 33/60] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:50.404347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:50.497974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc764d-80a4-4dab-8659-a2c17700c0c1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:50.530606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a78ae-7e5a-48ee-becf-d6f57178850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:50.531204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a78ae-7e5a-48ee-becf-d6f57178850e None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:50.531478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a78ae-7e5a-48ee-becf-d6f57178850e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:50.531748 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a78ae-7e5a-48ee-becf-d6f57178850e None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:50.532419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a78ae-7e5a-48ee-becf-d6f57178850e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:50.533838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a78ae-7e5a-48ee-becf-d6f57178850e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:50.612744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 28/61] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:02:50 2022] POST /identity/v3/users => generated 275 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:53.725857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3781794-7e88-4e48-b545-58d7c58662e8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:53.725857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3781794-7e88-4e48-b545-58d7c58662e8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:53.725857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3781794-7e88-4e48-b545-58d7c58662e8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:53.725857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 34/62] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:53.731965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eeba4-712d-48d3-889e-0df1ab1aa5cc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:53.732816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eeba4-712d-48d3-889e-0df1ab1aa5cc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:53.733299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eeba4-712d-48d3-889e-0df1ab1aa5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:53.752246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a18eeba4-712d-48d3-889e-0df1ab1aa5cc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:53.753761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a18eeba4-712d-48d3-889e-0df1ab1aa5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:53.782737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:53.828172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 29/63] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:53.849111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3781794-7e88-4e48-b545-58d7c58662e8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:53.894006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f455b-e5a0-4bc9-81df-8eefb5afcedb None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:53.894307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f455b-e5a0-4bc9-81df-8eefb5afcedb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:53.894563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43f455b-e5a0-4bc9-81df-8eefb5afcedb None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:53.895910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 30/64] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:02:53.904287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cab291-6cc0-4478-b096-a926cdf3291c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:53.905021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cab291-6cc0-4478-b096-a926cdf3291c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:53.905989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cab291-6cc0-4478-b096-a926cdf3291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:53.940269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-02cab291-6cc0-4478-b096-a926cdf3291c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:53.942059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-02cab291-6cc0-4478-b096-a926cdf3291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:53.996716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3781794-7e88-4e48-b545-58d7c58662e8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.034429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.035047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.035409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 31/65] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:54.035575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.036105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.037090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70bd2-1aca-49da-91ee-5dbbf0f6ad58 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.037600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70bd2-1aca-49da-91ee-5dbbf0f6ad58 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.038164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70bd2-1aca-49da-91ee-5dbbf0f6ad58 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.044262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 32/66] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:54.045692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.046176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.046663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.047546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:54.052302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:54.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:54.054209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 35/67] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:54.060386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.065496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.071834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:54.073656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:54.145473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 33/68] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:54.152016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.176150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a066a-282b-40c6-a1f4-833866503af7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.197921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.221204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.221939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.222081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.222353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.223011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:54.280142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:54.280142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:54.280142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 36/69] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:54.280142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.296437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ca2f9-9659-439e-adee-606da3be1237 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.316494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.338106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.338771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.339055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.339326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.350217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:54.355739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:54.356670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:02:54.357464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:02:54.358063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a7dd8-05d1-4f9a-b4af-829070d7b8ff None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.358140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 34/70] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:54.358140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.392776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f34de-1600-4e20-a932-736a6001178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.393415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f34de-1600-4e20-a932-736a6001178c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.393705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f34de-1600-4e20-a932-736a6001178c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.393982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f34de-1600-4e20-a932-736a6001178c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.405047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f34de-1600-4e20-a932-736a6001178c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-ff3f34de-1600-4e20-a932-736a6001178c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:54.410235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:54.411195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:54.412126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 37/71] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:54.413804 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb94c34-4f34-487f-8302-f7fa4aa0a6b8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.414105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb94c34-4f34-487f-8302-f7fa4aa0a6b8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.414428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb94c34-4f34-487f-8302-f7fa4aa0a6b8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.415741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 38/72] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:54.419750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71ed8a-4a0a-4368-a6ea-656d99bba126 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.419750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71ed8a-4a0a-4368-a6ea-656d99bba126 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.419750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71ed8a-4a0a-4368-a6ea-656d99bba126 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.419750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 39/73] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:54.419750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71ed8a-4a0a-4368-a6ea-656d99bba126 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.455234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.470829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e7ee0-480b-4a3e-ae34-4ccaf59097ed None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.503809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.504473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.504756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.505033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.518874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71ed8a-4a0a-4368-a6ea-656d99bba126 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.520612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:54.522066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:54.523680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 35/74] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:54.525115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.547774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.556339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.556942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.557217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.557483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.558133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:54.558430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:54.559789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d4ede5-543a-4806-9568-891d33f577de None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:54.567445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 40/75] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:54.568681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e1c3-a1e6-40c2-9769-1083a563c576 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.568968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e1c3-a1e6-40c2-9769-1083a563c576 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.569244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b3e1c3-a1e6-40c2-9769-1083a563c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.587422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-51b3e1c3-a1e6-40c2-9769-1083a563c576 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:54.588664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-51b3e1c3-a1e6-40c2-9769-1083a563c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:54.625499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f805b-e152-41e3-ad1d-a1786bd93195 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.653478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 41/76] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:54.654957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.655247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.655529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.662932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.663540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.663823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.664143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.664837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:54.665220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:54.666602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:54.674581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 36/77] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:54.675376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.682687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.683728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:54.684972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:54.752463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 42/78] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:54.753310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.794326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cf12-cc66-4d22-9202-276ae8ea3387 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.801933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.828517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.829127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.829407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.829679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.852554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caaf0df-5c4d-4c90-8fcf-07b55ef8cb4b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.885131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:54.885752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:54.886029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:54.886300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:54.887405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:54.888892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:54.890457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 43/79] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:54.895047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:54.897476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:54.898960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:54.901575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:54.987943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a57154-5cd4-42df-960c-20c42dff96da None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:54.994921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 37/80] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:02:54.996711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.004949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.023297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.023958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.024258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.024529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.025732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.027495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.029980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 44/81] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:55.030567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.101629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80c12cb-a5cd-4ad5-bd67-4658eb6c7347 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.103907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.132928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d451e5-4c08-4a32-b03b-595db5ee6b11 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.135414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.136109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.136393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.136667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.137801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.139276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.140889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 38/82] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:54 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:55.141653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.168484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.169239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.169525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.169806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.181282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:55.186476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:55.187494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 45/83] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:55.188344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.234690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.244782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c4e7b-ad6f-486a-8865-dd4bc78b92e9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.277295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.277897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.278181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.278454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.286334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f12b-96c1-45f7-9dd9-644f26a78435 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.319715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.320687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.321418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.321702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.321975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.323099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.324586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.326226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 46/84] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:55.326952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.345716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.347168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.427431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df994d-012a-4e85-8bcd-ec481b5908c8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.440599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.445230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 39/85] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:02:55.446165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.461306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41017d46-c546-4853-9787-388042fa3997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.461951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41017d46-c546-4853-9787-388042fa3997 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.462277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41017d46-c546-4853-9787-388042fa3997 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.462553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41017d46-c546-4853-9787-388042fa3997 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.463197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41017d46-c546-4853-9787-388042fa3997 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.464682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41017d46-c546-4853-9787-388042fa3997 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.514331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.542925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ec885-053b-49e2-9740-0550e26cec4b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.574693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.575290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.575568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.575838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.591135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 47/86] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:02:55 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:02:55.591964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41017d46-c546-4853-9787-388042fa3997 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.653183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.654602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.704729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41017d46-c546-4853-9787-388042fa3997 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.738592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.742883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.743372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.743656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.743933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.744684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:55.744995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.746387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.754913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 48/87] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:55.758244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.779716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 40/88] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:02:55.780724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.793052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:55.891516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b053c3dd-c8e9-42e2-a8ab-6523f57b0b70 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.915398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35942-ce84-4f61-a17c-3dce9da09655 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:55.938050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.938671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.939514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.939796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.940879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=67dbb8e285ad48eaa50f92f1a4327253) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:55.941196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.943488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.951933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 49/89] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/role_assignments?user.id=67dbb8e285ad48eaa50f92f1a4327253&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:55.952900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:55.955043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:55.956103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:55.957435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:55.957870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:55.959412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:55.961123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:55.964920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 41/90] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:55.966415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:56.056716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:56.084507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:56.091017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa1d9f-9158-4f05-94c2-f814b8066da3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:56.131859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:56.132810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:56.133789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:56.134328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:56.136751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:56.142670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:56.142670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 50/91] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:56.142670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:56.200700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2346fe1-897d-43bd-a9bc-418adca53bc0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:56.272760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:56.273694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:56.274165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:56.274599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:56.276698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:56.278414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:56.280682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 42/92] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:55 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:56.288104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:56.308027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77df0ef-79cb-4a72-b737-96f6f2ce0ac8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:56.346505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:56.347286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:56.347661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:56.348061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:56.348910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=0b3cf49f311d4c78b75ab2d7b48164db) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:56.349470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:56.356022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:56.358715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 51/93] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:56 2022] GET /identity/v3/role_assignments?user.id=0b3cf49f311d4c78b75ab2d7b48164db&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:56.359484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:56.448392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:56.531281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33416e-3070-4969-9802-02b45d08c8ba None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:56.559051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbcd15-db66-4971-8bc6-31f9835c37fc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:56.578012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:56.579507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:56.580171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:56.580595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:56.580965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:56.581835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] RBAC: Adding query filter params (role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=67dbb8e285ad48eaa50f92f1a4327253) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:56.584896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:56.584896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:56.591367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 52/94] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:56 2022] GET /identity/v3/role_assignments?user.id=67dbb8e285ad48eaa50f92f1a4327253&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=22011eb257084dc58e55ad842a3a3292 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:56.647136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:56.648040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:56.648468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:56.648946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:56.764185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:56.766012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:56.907721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 43/95] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:02:56.916142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:57.044417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:57.058550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b2db5-0265-4d72-bc6c-561bac767282 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:57.099781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0813b1-fa54-4e3e-9a94-6113ed871196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:57.100787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0813b1-fa54-4e3e-9a94-6113ed871196 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:57.101269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0813b1-fa54-4e3e-9a94-6113ed871196 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:57.101780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0813b1-fa54-4e3e-9a94-6113ed871196 None admin] PATH_INFO: `/v3/projects/9a9895ceead1480e9601d21898212b29` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:57.108697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0813b1-fa54-4e3e-9a94-6113ed871196 None admin] RBAC: Authorizing `identity:get_project(project_id=9a9895ceead1480e9601d21898212b29)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:57.110468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0813b1-fa54-4e3e-9a94-6113ed871196 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:57.112726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 53/96] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:56 2022] GET /identity/v3/projects/9a9895ceead1480e9601d21898212b29 => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:57.120646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:57.132536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:57.267851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b17628-3657-4e76-825f-389f1fa3f723 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:57.335494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:57.336151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:57.336970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:57.337280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:57.338196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=9a9895ceead1480e9601d21898212b29, user.id=67dbb8e285ad48eaa50f92f1a4327253) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:57.338512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:57.339977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f2e53-c5d6-432e-b4de-48f577e388dd None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:57.349002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 44/97] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:02:57 2022] GET /identity/v3/role_assignments?user.id=67dbb8e285ad48eaa50f92f1a4327253&scope.project.id=9a9895ceead1480e9601d21898212b29&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:58.376740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c51d3-ba60-42f4-8661-8b414149afdc None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:58.377185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c51d3-ba60-42f4-8661-8b414149afdc None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:58.377382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c51d3-ba60-42f4-8661-8b414149afdc None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:58.378782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 54/98] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:58.385610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:58.386048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:58.386144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:58.415099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:58.416333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:58.518447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 45/99] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:58.565511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c51d3-ba60-42f4-8661-8b414149afdc None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:58.688841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c51d3-ba60-42f4-8661-8b414149afdc None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:58.724356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c12b-190f-44d9-acf5-56130362716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:58.725006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c12b-190f-44d9-acf5-56130362716a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:58.725289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c12b-190f-44d9-acf5-56130362716a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:58.725565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c12b-190f-44d9-acf5-56130362716a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:58.737967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c12b-190f-44d9-acf5-56130362716a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-1473c12b-190f-44d9-acf5-56130362716a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:58.743590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:58.744470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:58.745245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 55/100] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:02:58.751146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:58.753373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:58.794824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5013942-4f51-4407-8e49-6fa4bf5540f3 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:58.795164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5013942-4f51-4407-8e49-6fa4bf5540f3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:58.795409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5013942-4f51-4407-8e49-6fa4bf5540f3 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:58.796816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 56/101] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:02:58.802007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:58.802329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:58.802588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:58.838115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:58.840976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:58.872942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:58.923696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c84c3-5aa5-46ed-b923-be4d4d418f52 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:58.939924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 57/102] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:02:58.974241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.000169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.001147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.001649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.002115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.002983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:59.003480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:59.006243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:59.026220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 46/103] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:02:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:59.056111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.121783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc81dcc-0625-4162-9477-ba8944dac1bf None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:59.172144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:59.173427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.174178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.174564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.174904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.194142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:02:59.199564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:02:59.200464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:02:59.201241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:02:59.212021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 58/104] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:02:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:02:59.220023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.236488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:59.276431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca5b33-36b3-41d3-a71b-b1617483ccfc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:59.381475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741609b-793c-423b-8f75-03aad58f95b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.382137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741609b-793c-423b-8f75-03aad58f95b2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.382418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741609b-793c-423b-8f75-03aad58f95b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.382695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741609b-793c-423b-8f75-03aad58f95b2 None admin] PATH_INFO: `/v3/users/4434c63390484f2fa523004b4fe0e5d0` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.389105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4102f016-6039-4ac7-9e3e-334775c2409e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:59.413257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741609b-793c-423b-8f75-03aad58f95b2 None admin] RBAC: Authorizing `identity:get_user(user_id=4434c63390484f2fa523004b4fe0e5d0)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:59.414793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741609b-793c-423b-8f75-03aad58f95b2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:59.416939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 47/105] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/users/4434c63390484f2fa523004b4fe0e5d0 => generated 289 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:02:59.417980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730eee9-bdfd-47fa-9284-66a4ba831f1a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.418263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730eee9-bdfd-47fa-9284-66a4ba831f1a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.418543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730eee9-bdfd-47fa-9284-66a4ba831f1a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.419876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 48/106] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:59.426032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730eee9-bdfd-47fa-9284-66a4ba831f1a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.434301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.435173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.435627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.436103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.436990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:02:59.437465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:59.439651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d6dde-e184-4583-af86-a4b6e8b89fe7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:59.450345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 59/107] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:59.452111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7aa81-458c-46a3-ab13-2b4c0b1abd0b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.452595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7aa81-458c-46a3-ab13-2b4c0b1abd0b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.453129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7aa81-458c-46a3-ab13-2b4c0b1abd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.472664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e5b7aa81-458c-46a3-ab13-2b4c0b1abd0b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:59.475043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e5b7aa81-458c-46a3-ab13-2b4c0b1abd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:59.544941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730eee9-bdfd-47fa-9284-66a4ba831f1a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:59.560828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:59.567744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 60/108] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:59.569745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:59.570218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a94ac5-83ab-4a31-a716-0d3a2d23f899 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.572454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a94ac5-83ab-4a31-a716-0d3a2d23f899 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.573027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a94ac5-83ab-4a31-a716-0d3a2d23f899 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.575209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 61/109] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:02:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:02:59.576274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a94ac5-83ab-4a31-a716-0d3a2d23f899 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.586419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.587036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.587320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.587600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.661713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:59.663242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:59.718370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a94ac5-83ab-4a31-a716-0d3a2d23f899 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:59.762771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e1ba9-4277-4f52-b4ba-7d66ee43ffab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.763750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e1ba9-4277-4f52-b4ba-7d66ee43ffab None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.764376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e1ba9-4277-4f52-b4ba-7d66ee43ffab None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.765202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e1ba9-4277-4f52-b4ba-7d66ee43ffab None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.766989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e1ba9-4277-4f52-b4ba-7d66ee43ffab None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:59.769283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312e1ba9-4277-4f52-b4ba-7d66ee43ffab None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:59.771694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 62/110] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:59.773284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a726-6b16-4641-871f-da2681810380 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.773875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a726-6b16-4641-871f-da2681810380 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.774459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a726-6b16-4641-871f-da2681810380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.774907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 49/111] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:02:59.775751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.797146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4b82a726-6b16-4641-871f-da2681810380 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:02:59.798721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4b82a726-6b16-4641-871f-da2681810380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:02:59.801959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:59.866409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:02:59.880269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 63/112] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:02:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:02:59.881342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a726-6b16-4641-871f-da2681810380 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:02:59.899582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810f21a-29f0-4a41-8759-b529b7c6e87d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:02:59.938393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:02:59.939029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:02:59.939311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:02:59.939586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:02:59.940710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:02:59.942174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:02:59.943737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 50/113] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:02:59.944530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.004409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a726-6b16-4641-871f-da2681810380 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.041076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.041572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:00.041747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.041985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.042486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.043824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:00.045325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:00.052107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 64/114] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:00.052107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.057878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e979eb-9a1c-484e-a9c3-622f30f0b6d0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.095622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639c686d-0552-4aa3-9930-543f5a250add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.096309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c686d-0552-4aa3-9930-543f5a250add None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.096596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c686d-0552-4aa3-9930-543f5a250add None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.096875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c686d-0552-4aa3-9930-543f5a250add None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.098323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c686d-0552-4aa3-9930-543f5a250add None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:00.099778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639c686d-0552-4aa3-9930-543f5a250add None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:00.101764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 51/115] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:02:59 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:00.102940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcd326-413b-43f8-bbc3-ec0f062fcfa0 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.103230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcd326-413b-43f8-bbc3-ec0f062fcfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.103509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcd326-413b-43f8-bbc3-ec0f062fcfa0 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.104739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 52/116] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:00.105428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcd326-413b-43f8-bbc3-ec0f062fcfa0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.127346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:00.163291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c5d2f-a05d-4b2f-a14f-36d65f63b48e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.198171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.198799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.199080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.199540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.208366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcd326-413b-43f8-bbc3-ec0f062fcfa0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.209146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:00.214790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:00.215732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:00.216520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 65/117] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:00.217152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.245782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.246390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.246681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.246962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.247609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:00.249101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:00.314312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcb403-5969-42e6-bef6-35238e2891e0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.339296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:00.340685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 53/118] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:03:00 2022] POST /identity/v3/projects => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:00.341357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.348794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bc36bd-4c9a-4368-b2ec-054bd27da665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.349493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc36bd-4c9a-4368-b2ec-054bd27da665 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.349778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc36bd-4c9a-4368-b2ec-054bd27da665 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.350132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc36bd-4c9a-4368-b2ec-054bd27da665 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078/users/67dbb8e285ad48eaa50f92f1a4327253/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.353728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc36bd-4c9a-4368-b2ec-054bd27da665 None admin] RBAC: Authorizing `identity:create_grant(project_id=d858077827a543fd96cbd471437ac078, user_id=67dbb8e285ad48eaa50f92f1a4327253, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:00.354654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:00.355974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bc36bd-4c9a-4368-b2ec-054bd27da665 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:00.433983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 66/119] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:00 2022] PUT /identity/v3/projects/d858077827a543fd96cbd471437ac078/users/67dbb8e285ad48eaa50f92f1a4327253/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:00.436643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7f000-b251-4417-9cc8-b5da14e90d41 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.436643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7f000-b251-4417-9cc8-b5da14e90d41 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.436643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7f000-b251-4417-9cc8-b5da14e90d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.457022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a1e7f000-b251-4417-9cc8-b5da14e90d41 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:00.458394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a1e7f000-b251-4417-9cc8-b5da14e90d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:00.470732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55037f1a-03f2-4e43-8111-f9f03102b718 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.516255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.516922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.517208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.517494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.518248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] RBAC: Adding query filter params (role.id=fa67919b5894446a8db1460851778f9d, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=4434c63390484f2fa523004b4fe0e5d0) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:00.518567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:00.520993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc3e1d6-cffb-49e9-bc0f-c3efda7ee4d3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:00.529033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 54/120] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:00 2022] GET /identity/v3/role_assignments?user.id=4434c63390484f2fa523004b4fe0e5d0&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=fa67919b5894446a8db1460851778f9d => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:00.530328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeeeed-22c1-477d-b864-b47411f3eb73 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.530621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeeeed-22c1-477d-b864-b47411f3eb73 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.530901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caeeeed-22c1-477d-b864-b47411f3eb73 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.532979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 55/121] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:00.532979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeeeed-22c1-477d-b864-b47411f3eb73 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.567639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 67/122] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:00.569358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.569788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.570203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.593520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:00.595019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:00.639052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:00.673623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caeeeed-22c1-477d-b864-b47411f3eb73 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.690040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 68/123] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:00.691459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.722901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.724291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.724628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.724904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.725750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:00.726212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:00.728881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:00.743949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 56/124] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:00.744919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:00.758865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:00.850810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc6842-fb7d-4fd6-b671-d3a2ce8e7cca None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.915292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:00.916339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:00.916872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:00.917533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:00.935576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:00.937733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d5c79-0ecb-4a2b-9b5c-b0eb9aab5c8f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:00.946295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:00.947219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:00.948025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:00.954776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 69/125] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:00.956022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.021751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.022507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.022837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.023152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.029101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:01.049150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:01.055378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:01.072346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 57/126] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:01.073226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.094327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b700ecd-e2a1-4d6c-a469-beaa38c95bbd None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.103441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:01.132357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.134027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.134521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.135101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.142023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.143723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.146532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 70/127] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:00 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:01.153159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.322116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91695a52-e420-497e-9596-28f82c342be9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.327017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dcebe-05bc-4349-a724-8f7fc427121c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.375217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.376097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.376570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.377024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.377872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:01.378372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.382937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.395440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 71/128] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:01.397081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.417191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.418129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.418599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.419052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.419914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:01.420487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.422063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.443683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 58/129] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:01.444782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.480362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:01.567806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d5558-dd8a-42f8-95fd-e039ab40448a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.605085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:01.609430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.611298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.612642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.617258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.617258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.617258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.622167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 72/130] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:01 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:01.624139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.673043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287eeae6-da2c-4fc0-a25e-ba4409def19a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.719599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.720380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.720732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.721080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.724044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.726070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.728894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 59/131] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:01 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:01.730040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:01.848299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0ce21-97ab-4fda-8974-914410de9112 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.863115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d84fcd-0fc4-4b0e-b8e3-6dd12f8fe622 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:01.872464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:01.881686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:01.903777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.904480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.904759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.905033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078/users/0b3cf49f311d4c78b75ab2d7b48164db/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.909947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] RBAC: Authorizing `identity:create_grant(project_id=d858077827a543fd96cbd471437ac078, user_id=0b3cf49f311d4c78b75ab2d7b48164db, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.911396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.923016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:01.923647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:01.923936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:01.924223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:01.925359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:01.926863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:01.932089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 73/132] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:01 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:01.933479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:02.040429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7601bac-dd45-429b-9688-7ad94e34bf3e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:02.076215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 60/133] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:01 2022] PUT /identity/v3/projects/d858077827a543fd96cbd471437ac078/users/0b3cf49f311d4c78b75ab2d7b48164db/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:02.077341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:02.107188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:02.107831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:02.108142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:02.108411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] PATH_INFO: `/v3/projects/9a9895ceead1480e9601d21898212b29` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:02.109548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] RBAC: Authorizing `identity:get_project(project_id=9a9895ceead1480e9601d21898212b29)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:02.112054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:02.112753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 74/134] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:01 2022] GET /identity/v3/projects/9a9895ceead1480e9601d21898212b29 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:02.118627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:02.164529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:02.231155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a34aa-cb71-40f4-bcb2-805a4c865f1f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:02.269267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22994a-81bd-4275-b82f-3a68e3265661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:02.270094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22994a-81bd-4275-b82f-3a68e3265661 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:02.270464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22994a-81bd-4275-b82f-3a68e3265661 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:02.270834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22994a-81bd-4275-b82f-3a68e3265661 None admin] PATH_INFO: `/v3/projects/9a9895ceead1480e9601d21898212b29/users/67dbb8e285ad48eaa50f92f1a4327253/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:02.278744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22994a-81bd-4275-b82f-3a68e3265661 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a9895ceead1480e9601d21898212b29, user_id=67dbb8e285ad48eaa50f92f1a4327253, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:02.281426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22994a-81bd-4275-b82f-3a68e3265661 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:02.303350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cac3a9-b4b4-4314-bea6-553d7412a90b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:02.345014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:02.345811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:02.346190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:02.346621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:02.347860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:02.349437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:02.356204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 61/135] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:02 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:02.368592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:02.374143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 75/136] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:02 2022] PUT /identity/v3/projects/9a9895ceead1480e9601d21898212b29/users/67dbb8e285ad48eaa50f92f1a4327253/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:03:02.451036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:02.510545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3ace-1c70-402b-b291-5112726e7d36 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:02.547261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af7d50-1b1a-4c34-b26c-6109234b6516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:02.547936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af7d50-1b1a-4c34-b26c-6109234b6516 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:02.548252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af7d50-1b1a-4c34-b26c-6109234b6516 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:02.548526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af7d50-1b1a-4c34-b26c-6109234b6516 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078/users/67dbb8e285ad48eaa50f92f1a4327253/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:02.550749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af7d50-1b1a-4c34-b26c-6109234b6516 None admin] RBAC: Authorizing `identity:create_grant(project_id=d858077827a543fd96cbd471437ac078, user_id=67dbb8e285ad48eaa50f92f1a4327253, role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:02.552253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af7d50-1b1a-4c34-b26c-6109234b6516 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:02.660076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 62/137] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:02 2022] PUT /identity/v3/projects/d858077827a543fd96cbd471437ac078/users/67dbb8e285ad48eaa50f92f1a4327253/roles/22011eb257084dc58e55ad842a3a3292 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:02.741887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2e54-117c-4157-bcbd-8d958b255e8d None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:02.742442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2e54-117c-4157-bcbd-8d958b255e8d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:02.742442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d2e54-117c-4157-bcbd-8d958b255e8d None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:02.744349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 76/138] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:02.750134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08c2cb-e5ca-4d6a-b6c6-096b213b98ce None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:02.750405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08c2cb-e5ca-4d6a-b6c6-096b213b98ce None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:02.750682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08c2cb-e5ca-4d6a-b6c6-096b213b98ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:02.772768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cd08c2cb-e5ca-4d6a-b6c6-096b213b98ce None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:02.774001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cd08c2cb-e5ca-4d6a-b6c6-096b213b98ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:02.866040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 63/139] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:02.913790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2e54-117c-4157-bcbd-8d958b255e8d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:03.041624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:03.047798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d2e54-117c-4157-bcbd-8d958b255e8d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:03.081355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea56992-f9db-4b66-a0fa-7b3cb01e2061 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.081642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea56992-f9db-4b66-a0fa-7b3cb01e2061 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.081904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea56992-f9db-4b66-a0fa-7b3cb01e2061 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.083429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 64/140] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:03.094775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.094775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.094775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.101969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:03.102252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.102590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.102858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.117229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:03.119323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:03.125024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:03.130001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:03.130878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:03.131744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 77/141] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:03.139461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:03.172583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:03.205276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 65/142] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:03.250046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:03.344088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6b9697-8b52-4e8c-95dc-d6c7d6f85311 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:03.388774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:03.389893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.390386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.390862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.391743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:03.392337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:03.393959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521e9bf-bd11-4013-a45e-8810001418d8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:03.404212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 78/143] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:03.405728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65943a-50d7-471e-b2e1-dfded07aa458 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.406209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65943a-50d7-471e-b2e1-dfded07aa458 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.406713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65943a-50d7-471e-b2e1-dfded07aa458 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.408177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 79/144] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:03.410234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65943a-50d7-471e-b2e1-dfded07aa458 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:03.452385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:03.460479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c034b-2618-4354-a35f-10eaf4e09f1e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:03.498932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8de8a-19c1-4d3e-aae0-cc55092b2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:03.499575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8de8a-19c1-4d3e-aae0-cc55092b2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.500098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8de8a-19c1-4d3e-aae0-cc55092b2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.501909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8de8a-19c1-4d3e-aae0-cc55092b2fb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.503463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8de8a-19c1-4d3e-aae0-cc55092b2fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:03.505434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8de8a-19c1-4d3e-aae0-cc55092b2fb4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:03.508165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 66/145] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:03.509439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.516379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.516683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.558019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:03.559258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:03.607781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65943a-50d7-471e-b2e1-dfded07aa458 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:03.628435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:03.648345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:03.649004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.649349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.649626 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.650749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:03.653121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:03.658684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 80/146] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:03 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:03.665561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:03.682356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 67/147] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:03.683226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:03.791451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:03.810813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84b268-2807-425c-ab4c-9870e677e869 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:03.860974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:03.861922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.862417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.862895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.863745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:03.865436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:03.866152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6e9f2-6ee6-430c-8754-1b156a29f46a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:03.904899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:03.905647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:03.905936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:03.906215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:03.954827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:03.983599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:03.985154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:04.034541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 81/148] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:03 2022] POST /identity/v3/users => generated 283 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:04.035679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:04.128200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 68/149] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:04.144349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:04.230516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f96969-6ed6-4a66-96fb-eae5d42a9153 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:04.269005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:04.288550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58993aa5-5566-42b6-a912-348e92b5cbf8 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:04.313953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:04.314609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.314887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.315160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.330058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:04.330719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.331003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.331283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.333198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:04.334643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:04.339901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 69/150] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:04 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:04.340519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:04.345995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:04.346867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:04.348246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 82/151] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:04.349151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:04.358637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:04.400968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:04.470193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9e3f7-8b83-4483-89cb-d2cd8889f400 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:04.507467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:04.508207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.508502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.508850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.509512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=67dbb8e285ad48eaa50f92f1a4327253) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:04.509831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:04.511295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:04.519215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073657bd-0a48-48ee-94c5-eab1bc399377 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:04.524614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 70/152] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:04 2022] GET /identity/v3/role_assignments?user.id=67dbb8e285ad48eaa50f92f1a4327253&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:04.559616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:04.560385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.560751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.561095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.561848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:04.562221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:04.563745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:04.573791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 83/153] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:04.579669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:04.584517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:04.701984 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ef6ee-51a5-4551-99c3-1d0a73959c4b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:04.741028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13bb4-c39d-4b1e-90b4-aa3cee1c4cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:04.741687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13bb4-c39d-4b1e-90b4-aa3cee1c4cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.741975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13bb4-c39d-4b1e-90b4-aa3cee1c4cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.742315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13bb4-c39d-4b1e-90b4-aa3cee1c4cd7 None admin] PATH_INFO: `/v3/users/4434c63390484f2fa523004b4fe0e5d0` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.743731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13bb4-c39d-4b1e-90b4-aa3cee1c4cd7 None admin] RBAC: Authorizing `identity:get_user(user_id=4434c63390484f2fa523004b4fe0e5d0)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:04.745280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13bb4-c39d-4b1e-90b4-aa3cee1c4cd7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:04.747171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 71/154] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:04 2022] GET /identity/v3/users/4434c63390484f2fa523004b4fe0e5d0 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:04.753636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:04.826793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:04.862713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3fe2ee-6263-476e-b027-9e23b4c12f5e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:04.889691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9e365-8e03-4e3f-a062-9910b8790652 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.889993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9e365-8e03-4e3f-a062-9910b8790652 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.890321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f9e365-8e03-4e3f-a062-9910b8790652 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.891840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 72/155] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:04.901462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.901749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.902021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.922041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:04.922771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:04.923065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:04.923363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:04.924683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:04.925047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:04.926142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:04.926664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:04.927877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 84/156] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:04 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:04.934161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:05.037162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 73/157] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:05.061935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:05.065282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:05.093753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf5450e-44a6-4d06-91aa-683129d6c992 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:05.140723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:05.145484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.145484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.145484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078/users/4434c63390484f2fa523004b4fe0e5d0/roles/fa67919b5894446a8db1460851778f9d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.145484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] RBAC: Authorizing `identity:create_grant(project_id=d858077827a543fd96cbd471437ac078, user_id=4434c63390484f2fa523004b4fe0e5d0, role_id=fa67919b5894446a8db1460851778f9d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:05.146833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:05.152316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:05.190810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c69ab2-074f-4aa5-91c6-1eae047a1d6b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:05.231232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:05.231868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.232198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.232473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.244680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:05.247554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 85/158] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:04 2022] PUT /identity/v3/projects/d858077827a543fd96cbd471437ac078/users/4434c63390484f2fa523004b4fe0e5d0/roles/fa67919b5894446a8db1460851778f9d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:05.250635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:05.251550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:05.252905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 74/159] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:05.259285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:05.399647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47df08-5e24-46c4-976b-b8db52b5e831 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:05.449291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:05.449920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.450204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.450471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.451138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:05.451442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:05.452874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501c29a-f81f-43cf-aa60-ec84bff4d4a8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:05.464366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 86/160] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:05.474055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:05.553915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:05.597468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4713f296-dca6-4fcc-b8eb-66866b0f88be None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:05.605765 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45311-b54b-4e86-b95e-7dc13d0fdc07 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.606052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45311-b54b-4e86-b95e-7dc13d0fdc07 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.606313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d45311-b54b-4e86-b95e-7dc13d0fdc07 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.607779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 87/161] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:05.615703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763066-4386-4590-b846-7b1dfba5257a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.615977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763066-4386-4590-b846-7b1dfba5257a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.616278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93763066-4386-4590-b846-7b1dfba5257a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.644904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-93763066-4386-4590-b846-7b1dfba5257a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:05.646030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-93763066-4386-4590-b846-7b1dfba5257a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:05.650727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad688617-45c0-462a-9331-b8f4e711e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:05.651630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad688617-45c0-462a-9331-b8f4e711e752 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.652147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad688617-45c0-462a-9331-b8f4e711e752 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.652611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad688617-45c0-462a-9331-b8f4e711e752 None admin] PATH_INFO: `/v3/users/0b3cf49f311d4c78b75ab2d7b48164db` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.654000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad688617-45c0-462a-9331-b8f4e711e752 None admin] RBAC: Authorizing `identity:get_user(user_id=0b3cf49f311d4c78b75ab2d7b48164db)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:05.656174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad688617-45c0-462a-9331-b8f4e711e752 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:05.658662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 75/162] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:05 2022] GET /identity/v3/users/0b3cf49f311d4c78b75ab2d7b48164db => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:05.660366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7024fe-7c20-4c3c-842b-12f6f15b4729 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.660851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7024fe-7c20-4c3c-842b-12f6f15b4729 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.661334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7024fe-7c20-4c3c-842b-12f6f15b4729 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.663122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 76/163] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:05.666401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7024fe-7c20-4c3c-842b-12f6f15b4729 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:05.717045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:05.751845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 88/164] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:05.754416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062823d-9492-4b17-834e-432a4451247d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.756406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062823d-9492-4b17-834e-432a4451247d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.757702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062823d-9492-4b17-834e-432a4451247d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.795307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a062823d-9492-4b17-834e-432a4451247d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:05.798892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a062823d-9492-4b17-834e-432a4451247d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:05.839365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7024fe-7c20-4c3c-842b-12f6f15b4729 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:05.902122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:05.915226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:05.917192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:05.917677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:05.919126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:05.943542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 89/165] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:05.947090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062823d-9492-4b17-834e-432a4451247d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.083468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:06.087886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:06.184733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062823d-9492-4b17-834e-432a4451247d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:06.225330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:06.258193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:06.258969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:06.261166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:06.261520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:06.266401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 77/166] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:06.267850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.284782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:06.290652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:06.291546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:06.292617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:06.296407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 90/167] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:06.297333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.311897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:06.496200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339eccb-8217-4341-a8ad-d344fd771d02 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:06.547123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e36f82-7c07-43c9-b5cd-c05e58877fce None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:06.602783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:06.603416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:06.603693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:06.603964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:06.612946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:06.614428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:06.616446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 91/168] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:06 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:06.616716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:06.617212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.617969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:06.618242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:06.624032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:06.669235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:06.673487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:06.674161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:06.675016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:06.676445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 78/169] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:06.683149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.733931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f231c1d-f8e5-4546-a257-90323ee6beff None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:06.779090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:06.779621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:06.779895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:06.780186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:06.783446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:06.783761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:06.785759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:06.796438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 92/170] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:06.797569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.868652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0d012-0a01-471d-b25d-84c9441af223 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:06.884730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:06.936712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:06.937331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:06.937610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:06.937882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:06.938614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=0b3cf49f311d4c78b75ab2d7b48164db) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:06.938919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:06.940388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:06.952109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 79/171] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:06 2022] GET /identity/v3/role_assignments?user.id=0b3cf49f311d4c78b75ab2d7b48164db&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:06.953036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:06.960256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50cedc-5cf0-4ce3-a01b-62fdbb42de33 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:06.984519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:07.026542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:07.027169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:07.027445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:07.027728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:07.028400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:07.028700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:07.030146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:07.039071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 93/172] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:07.045864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:07.150800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b02a7-f08f-4e9f-a607-d9b6930278f8 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:07.172513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280617a4-04d4-4047-83c1-d0f7cc70c63c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:07.216540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:07.217371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:07.217659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:07.217936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:07.219407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:07.220941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:07.222880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 94/173] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:07 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:07.229478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:07.232761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:07.233423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:07.233701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:07.233978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] PATH_INFO: `/v3/users/67dbb8e285ad48eaa50f92f1a4327253` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:07.235230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] RBAC: Authorizing `identity:get_user(user_id=67dbb8e285ad48eaa50f92f1a4327253)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:07.236769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:07.238426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 80/174] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:06 2022] GET /identity/v3/users/67dbb8e285ad48eaa50f92f1a4327253 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:07.244484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:07.290204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:07.291000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:07.362740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afbe69-28f9-4506-a6b7-751fa22b8fd5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:07.408642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:07.409266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:07.409545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:07.409812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:07.409914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf93639-6d62-4c5a-96c7-c12fd40d6447 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:07.483315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:07.483931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:07.484233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:07.484503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:07.543808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:07.545222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:07.618594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:07.620203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:07.623805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:07.671837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:07.725273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 95/175] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:07.731262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:07.821943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 81/176] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:07.836432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:07.864788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3358f6b9-7389-4cf5-be07-542933dccbac None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:07.922152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:07.922803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:07.923067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:07.923354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:07.932033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:07.932033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:07.933144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 96/177] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:07 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:07.938956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:07.951737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:07.982591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3419f0df-32e5-43a8-a542-c047df86b548 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.010600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.032717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9c317-d739-438c-9219-c328fa666046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.033679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c317-d739-438c-9219-c328fa666046 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.034191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c317-d739-438c-9219-c328fa666046 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.034660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9c317-d739-438c-9219-c328fa666046 None admin] PATH_INFO: `/v3/projects/9a9895ceead1480e9601d21898212b29` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.037245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9c317-d739-438c-9219-c328fa666046 None admin] RBAC: Authorizing `identity:get_project(project_id=9a9895ceead1480e9601d21898212b29)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:08.038912 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9c317-d739-438c-9219-c328fa666046 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:08.044914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 82/178] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:07 2022] GET /identity/v3/projects/9a9895ceead1480e9601d21898212b29 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:08.046376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f6f6-b0d4-456c-8f83-73d15f351d00 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.047839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f6f6-b0d4-456c-8f83-73d15f351d00 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.049053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f6f6-b0d4-456c-8f83-73d15f351d00 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.050766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 83/179] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:08.052837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f6f6-b0d4-456c-8f83-73d15f351d00 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.104871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff79ee83-87d4-4d51-8a6a-d2eb9b5cc565 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.143021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.144214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.145896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.146404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.148307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] RBAC: Adding query filter params (role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=67dbb8e285ad48eaa50f92f1a4327253) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:08.149568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:08.151212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a7eaf-2440-4882-abfd-62b24ea5d639 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:08.167423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 97/180] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:07 2022] GET /identity/v3/role_assignments?user.id=67dbb8e285ad48eaa50f92f1a4327253&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=22011eb257084dc58e55ad842a3a3292 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:08.170975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabe095-0e80-4988-b91d-063afc9f753d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.171545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabe095-0e80-4988-b91d-063afc9f753d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.174369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dabe095-0e80-4988-b91d-063afc9f753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.214327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5dabe095-0e80-4988-b91d-063afc9f753d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:08.217662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5dabe095-0e80-4988-b91d-063afc9f753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:08.272503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f6f6-b0d4-456c-8f83-73d15f351d00 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.339223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.339964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.340337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.340676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.341500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=9a9895ceead1480e9601d21898212b29, user.id=67dbb8e285ad48eaa50f92f1a4327253) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:08.341876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:08.343404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764adf05-2dcd-49d4-ad23-766fa225e049 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:08.350219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 84/181] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/role_assignments?user.id=67dbb8e285ad48eaa50f92f1a4327253&scope.project.id=9a9895ceead1480e9601d21898212b29&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:08.351570 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3329e7-22f5-4848-9f4f-55d5c8098ec2 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.357340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.360308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3329e7-22f5-4848-9f4f-55d5c8098ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.360754 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3329e7-22f5-4848-9f4f-55d5c8098ec2 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.368040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 85/182] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:08.368040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5d955-db01-4b5a-849b-4194d5875c3e None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.368040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5d955-db01-4b5a-849b-4194d5875c3e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.368040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5d955-db01-4b5a-849b-4194d5875c3e None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.368040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 86/183] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:08.368688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.368985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.369262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.377061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 98/184] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:08.378633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.379359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.379825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.401281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:08.404118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:08.426525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:08.427736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:08.494000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.514755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 87/185] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:08.515691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.527793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 99/186] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:08.550767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.645342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd4562-6605-4725-9e97-980f9f21f11e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.665157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bfd58-a24d-4f65-96fb-39abe910e9a8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.683192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.683820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.684166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.684397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.695007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:08.695175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:08.700519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:08.701393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:08.702204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 88/187] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:08.702899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.707100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.707862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.708167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.714195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.719791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:08.724516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:08.725449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:08.726280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 100/188] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:08.728301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.755723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.810262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0736fa8-6928-4126-b3e4-0ea3f82524f0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.829560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459d79e-24bd-4a49-921f-7899d510d6d3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.843553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.844208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.844485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.844758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.854996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:08.860199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:08.861117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:08.861940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 89/189] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:08.862767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.868033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:08.868639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.868914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.869185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.869845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:08.870141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:08.871504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b648f7-379e-4109-ada3-a3aa717bc735 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:08.879825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 101/190] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:08.885239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5256e-5c27-4c4d-897a-75ebb6436058 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:08.885430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5256e-5c27-4c4d-897a-75ebb6436058 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:08.885705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5256e-5c27-4c4d-897a-75ebb6436058 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:08.886971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 102/191] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:08.888185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5256e-5c27-4c4d-897a-75ebb6436058 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:08.940174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.966864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd9be5-5100-460e-8aa8-c84bd8189a47 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.969128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:08.988235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5256e-5c27-4c4d-897a-75ebb6436058 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:08.999902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.000598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.000941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.001164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.002605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:09.002908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.004311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.013014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 90/192] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:09.013803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.028159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.028770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.029045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.029311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.029963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:09.030266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.031657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b700-8b44-464a-9070-373aca0988df None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.039684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 103/193] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:09.044135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.044407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.044686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.063341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:09.064561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:09.110696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df775e-658c-48d8-9fc4-c0d766050311 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.129633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 104/194] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:09.130441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.151362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.152018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.152318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.152588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] PATH_INFO: `/v3/users/29636b2bb2f844d3af8818327af56c4a` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.167005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] RBAC: Authorizing `identity:get_user(user_id=29636b2bb2f844d3af8818327af56c4a)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.168488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.170123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 91/195] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/users/29636b2bb2f844d3af8818327af56c4a => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:09.170883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.178839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.233807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cd6f3-9745-4e7e-9db0-0a2f9dffd8bb None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.239796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.265984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.266605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.266883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.267154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] PATH_INFO: `/v3/users/29636b2bb2f844d3af8818327af56c4a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.268303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] RBAC: Authorizing `identity:get_user(user_id=29636b2bb2f844d3af8818327af56c4a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.269742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.271305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 105/196] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/users/29636b2bb2f844d3af8818327af56c4a => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:09.272077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.317229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d37e0-8722-45aa-bdc4-05bdbcbc2293 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.351423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.352873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.353161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.353433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] PATH_INFO: `/v3/users/4434c63390484f2fa523004b4fe0e5d0` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.354568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] RBAC: Authorizing `identity:get_user(user_id=4434c63390484f2fa523004b4fe0e5d0)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.356019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.357591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 92/197] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/users/4434c63390484f2fa523004b4fe0e5d0 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:09.358344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.375961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2e99de-733f-44fc-b5b9-e15cdf59d798 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.389796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.410919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.411540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.411814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.412194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.413419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.415385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.416977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 106/198] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:09.417725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.450271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.458689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb1b3a-0943-49c0-8e22-061a9794943f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.493906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.494440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.494714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.494982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.517221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40faf04-7a5b-4f43-bc71-ec6510c0ef68 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.549787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.550385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.550662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.550931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.561622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.563112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.598502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.616824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.618389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.652139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.660968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 93/199] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:09.661753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.715509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 107/200] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:09.720918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.761445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349dd9a4-083c-4efe-a241-facf1dfe7c29 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.794943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.795545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.795821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.796130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.799039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.818356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d4c1e-08a4-4200-91da-18b10745c75b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:09.851545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:09.852203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:09.852482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:09.852759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:09.853400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.853822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:09.854854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.862756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:09.864278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:09.951493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 108/201] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:09 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:09.952392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:09.967350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 94/202] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:09.968706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:10.077487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:10.084721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0d63b-609f-403c-ac5b-74253d427ed2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:10.085971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d407351-bc33-4327-b441-d82131c5d851 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:10.122327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:10.122965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:10.123243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:10.123516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:10.124673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:10.126110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:10.128863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 109/203] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:10.128863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:10.133874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:10.134688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:10.135346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:10.136296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:10.136296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:10.137257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:10.138699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:10.140375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 95/204] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:09 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:10.144878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:10.247382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38f33d-15a4-4ad3-aaf6-4d92cc67e9d3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:10.271716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a489149-7eaf-40a6-b42d-76b0bd5d4845 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:10.286717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:10.287354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:10.287631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:10.287903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:10.292175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:10.293171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:10.295021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 110/205] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:10 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:10.300417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:10.319196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:10.319804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:10.320112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:10.320410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:10.321134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=29636b2bb2f844d3af8818327af56c4a) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:10.321435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:10.322968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:10.336129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 96/206] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:10 2022] GET /identity/v3/role_assignments?user.id=29636b2bb2f844d3af8818327af56c4a&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:10.336129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:10.412583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:10.437773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:10.439416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30881c27-2fbe-4fd8-b34b-525ce7633c3c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:10.485026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:10.485643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:10.485924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:10.486192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:10.486924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] RBAC: Adding query filter params (role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=29636b2bb2f844d3af8818327af56c4a) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:10.487226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:10.488725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c823a-25b9-4b9a-8179-9e17a6298b15 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:10.511630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 111/207] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:10 2022] GET /identity/v3/role_assignments?user.id=29636b2bb2f844d3af8818327af56c4a&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=22011eb257084dc58e55ad842a3a3292 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:10.557446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858e69-c941-4111-bd4e-a59728d37bf2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:10.594641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:10.595165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:10.595438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:10.595705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:10.596458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] RBAC: Adding query filter params (role.id=fa67919b5894446a8db1460851778f9d, scope.project.id=d858077827a543fd96cbd471437ac078, user.id=4434c63390484f2fa523004b4fe0e5d0) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:10.596761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:10.598213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db804a94-4863-456c-ac0d-3f91da6d72df None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:10.605029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 97/208] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:10 2022] GET /identity/v3/role_assignments?user.id=4434c63390484f2fa523004b4fe0e5d0&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=fa67919b5894446a8db1460851778f9d => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:12.198983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca285fd-ec7f-48f0-b67f-ded0d88d52f0 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.200022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca285fd-ec7f-48f0-b67f-ded0d88d52f0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.200539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca285fd-ec7f-48f0-b67f-ded0d88d52f0 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.202306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 112/209] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:12.207817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f89c-75aa-4397-9176-b10b51f41462 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.208671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f89c-75aa-4397-9176-b10b51f41462 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.209166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be89f89c-75aa-4397-9176-b10b51f41462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.241279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-be89f89c-75aa-4397-9176-b10b51f41462 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:12.243113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-be89f89c-75aa-4397-9176-b10b51f41462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:12.315260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 98/210] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:12.339414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca285fd-ec7f-48f0-b67f-ded0d88d52f0 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:12.367927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1d74-2417-4c46-a6b0-f3633d2c98fc None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.368511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1d74-2417-4c46-a6b0-f3633d2c98fc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.369025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da1d74-2417-4c46-a6b0-f3633d2c98fc None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.371518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 99/211] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:12.376289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de356178-115d-4afa-b625-a7f71ee22911 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.376798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de356178-115d-4afa-b625-a7f71ee22911 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.377294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de356178-115d-4afa-b625-a7f71ee22911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.397125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-de356178-115d-4afa-b625-a7f71ee22911 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:12.398590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-de356178-115d-4afa-b625-a7f71ee22911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:12.413618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:12.459647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca285fd-ec7f-48f0-b67f-ded0d88d52f0 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:12.477068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 100/212] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:12.485995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d970ac2-58f8-4396-ad3f-b886fcfb3fa8 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.486503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d970ac2-58f8-4396-ad3f-b886fcfb3fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.487182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d970ac2-58f8-4396-ad3f-b886fcfb3fa8 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.489693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 101/213] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:12.495089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.495404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.495761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.499167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:12.500090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.500731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.501009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.505159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:12.514530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:12.514672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:12.515737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:12.520724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:12.521554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:12.522361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 113/214] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:12.524910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:12.600342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 102/215] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:12.601618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:12.633807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243cb485-84c0-4d63-b62b-87a7a5c4ee71 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:12.670829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:12.671465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.671730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.672099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.673241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:12.675066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:12.681409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 114/216] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:12.681409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:12.694704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:12.718175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70cbbf-3104-450d-bf26-82b84b2ecf3d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:12.747823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:12.759500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:12.760137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.760417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.760687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.761350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:12.761646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:12.763015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:12.771067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 103/217] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:12.771904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:12.790396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e166df-52cc-479c-99b5-79344e10a32b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:12.825728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c6adf-51fe-44e7-b212-866ca76932d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:12.826360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c6adf-51fe-44e7-b212-866ca76932d1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.826634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c6adf-51fe-44e7-b212-866ca76932d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.826904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c6adf-51fe-44e7-b212-866ca76932d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.836674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c6adf-51fe-44e7-b212-866ca76932d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-477c6adf-51fe-44e7-b212-866ca76932d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:12.841773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:12.842715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 115/218] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:12.848467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53130-1eaa-4203-943a-28fdda8bfee6 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.849730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53130-1eaa-4203-943a-28fdda8bfee6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.849730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd53130-1eaa-4203-943a-28fdda8bfee6 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.849730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 116/219] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:12.849730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53130-1eaa-4203-943a-28fdda8bfee6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:12.873313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623d728-4855-44b7-aba3-5a94baad4df1 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:12.906674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3a376-d4e6-4901-b046-5004bd71b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:12.907272 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a376-d4e6-4901-b046-5004bd71b93f None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.907546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a376-d4e6-4901-b046-5004bd71b93f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.907817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3a376-d4e6-4901-b046-5004bd71b93f None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.908489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3a376-d4e6-4901-b046-5004bd71b93f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:12.909914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3a376-d4e6-4901-b046-5004bd71b93f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:12.922572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:12.949066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd53130-1eaa-4203-943a-28fdda8bfee6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:12.971544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:12.981677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:12.982305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:12.982583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:12.982854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] PATH_INFO: `/v3/users/ef7609cee46046e18efe543352c08ba6` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:12.995893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] RBAC: Authorizing `identity:get_user(user_id=ef7609cee46046e18efe543352c08ba6)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:12.997415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:12.999019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 117/220] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:12 2022] GET /identity/v3/users/ef7609cee46046e18efe543352c08ba6 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:12.999770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.024029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 104/221] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:12 2022] POST /identity/v3/users => generated 297 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:13.024029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.024029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.024029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.038797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:13.040141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:13.109513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a252fe-44eb-42d3-a54c-7934b102fb2b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.122552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 105/222] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:13.123474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.157871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.158732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.159439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.159713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.160684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:13.161033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.163316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.176514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 118/223] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:13.177629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.229419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:13.244056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1de84a-366e-43bb-92ce-a63988c3d1ab None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.282585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:13.290672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.291723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.292263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.292752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.305766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed6a77-7cc0-4b2e-9716-f53f9182e64e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.348728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.350021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.350497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.350995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.363528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:13.369962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:13.370891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:13.371833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 119/224] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:13.375865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.389000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.390704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.499537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 106/225] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:13.500691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.500977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339e4a7-1640-4fa7-b93f-e313b76004f6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.503504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:13.536423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:13.543602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.544404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.544746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.545085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] PATH_INFO: `/v3/users/29636b2bb2f844d3af8818327af56c4a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.547267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] RBAC: Authorizing `identity:get_user(user_id=29636b2bb2f844d3af8818327af56c4a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.548790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.550655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 120/226] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/users/29636b2bb2f844d3af8818327af56c4a => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:13.556560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.627381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fa48f-ce8c-40ab-b199-7ad4f433b9a9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.667770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.668522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.668865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.669197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.670217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:13.670587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.672047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.681323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 107/227] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:13.682333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.696368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177255e-914a-4845-800d-d0b02ab596e0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.737803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.738513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.738787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.739056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.740249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.741811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.743623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 121/228] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:13.744541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.789908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:13.805177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:13.822494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc596d-449d-4f0d-8cda-366920dd490c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.857925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d71e-d520-4b70-a06d-247170ec512b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:13.871130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.872300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.872806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.873687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.875575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.877674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.880128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 108/229] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:13.881180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:13.901776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:13.902975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:13.903498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:13.904325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] PATH_INFO: `/v3/users/29636b2bb2f844d3af8818327af56c4a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:13.905941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] RBAC: Authorizing `identity:get_user(user_id=29636b2bb2f844d3af8818327af56c4a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:13.908193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:13.910046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 122/230] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/users/29636b2bb2f844d3af8818327af56c4a => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:13.911386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:14.012188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505c9a5-5a4c-4ce7-b658-4284cd534559 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:14.034525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c1804-8523-4189-b2ca-4f4f9b3f0e61 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:14.044313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:14.057908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:14.058917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:14.059679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:14.060369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:14.061116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=ef7609cee46046e18efe543352c08ba6) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:14.061435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:14.062932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:14.068744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 109/231] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:13 2022] GET /identity/v3/role_assignments?user.id=ef7609cee46046e18efe543352c08ba6&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:14.076050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:14.082473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:14.084332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:14.084772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:14.087430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608/users/29636b2bb2f844d3af8818327af56c4a/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:14.089989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0187a24b848949bcd600864be9608, user_id=29636b2bb2f844d3af8818327af56c4a, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:14.091669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:14.116287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:14.168235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 123/232] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:13 2022] PUT /identity/v3/projects/bcc0187a24b848949bcd600864be9608/users/29636b2bb2f844d3af8818327af56c4a/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:14.229812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c506551-fb5b-4121-96de-09bdf129d590 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:14.270315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d5de2-47f6-4360-9ed9-bf38faa00379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:14.271752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d5de2-47f6-4360-9ed9-bf38faa00379 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:14.272380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d5de2-47f6-4360-9ed9-bf38faa00379 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:14.273026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d5de2-47f6-4360-9ed9-bf38faa00379 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:14.275019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d5de2-47f6-4360-9ed9-bf38faa00379 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:14.276758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d5de2-47f6-4360-9ed9-bf38faa00379 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:14.279138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 110/233] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:14 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:14.284556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:14.404610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2f08e-a47f-4632-bc8b-b2a24594a2c1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:14.454276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a7788b-e59e-4d2c-80c2-d00213c5b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:14.454949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7788b-e59e-4d2c-80c2-d00213c5b389 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:14.455226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7788b-e59e-4d2c-80c2-d00213c5b389 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:14.455539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a7788b-e59e-4d2c-80c2-d00213c5b389 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608/users/29636b2bb2f844d3af8818327af56c4a/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:14.457818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7788b-e59e-4d2c-80c2-d00213c5b389 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0187a24b848949bcd600864be9608, user_id=29636b2bb2f844d3af8818327af56c4a, role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:14.459265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a7788b-e59e-4d2c-80c2-d00213c5b389 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:14.546131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 124/234] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:14 2022] PUT /identity/v3/projects/bcc0187a24b848949bcd600864be9608/users/29636b2bb2f844d3af8818327af56c4a/roles/22011eb257084dc58e55ad842a3a3292 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:03:15.372124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a943df8-5410-4fc3-834a-0875870a0207 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:15.372600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a943df8-5410-4fc3-834a-0875870a0207 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:15.372885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a943df8-5410-4fc3-834a-0875870a0207 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:15.374618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 111/235] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:15.387048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:15.387513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:15.387604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:15.411939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:15.413348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:15.443832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:15.497587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a27bdb-65cb-4aea-b93c-595dfc3c55cd None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:15.497851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a27bdb-65cb-4aea-b93c-595dfc3c55cd None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:15.498118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a27bdb-65cb-4aea-b93c-595dfc3c55cd None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:15.499576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 112/236] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:15.505429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 125/237] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:15.509020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb64be-6367-4def-866e-e1780142a69a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:15.509299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb64be-6367-4def-866e-e1780142a69a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:15.509566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb64be-6367-4def-866e-e1780142a69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:15.537043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-32eb64be-6367-4def-866e-e1780142a69a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:15.538213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-32eb64be-6367-4def-866e-e1780142a69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:15.586798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:15.638497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 113/238] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:15.683024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb64be-6367-4def-866e-e1780142a69a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:15.718398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:15.782785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7d615-ade5-44ad-82e5-86bcee1cb25e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:15.853324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb64be-6367-4def-866e-e1780142a69a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:15.857257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:15.857902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:15.858177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:15.858443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:15.864655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:15.866104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:15.867950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 126/239] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:15.874388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:15.904196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:15.904753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:15.905031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:15.905302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:15.923890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:15.924312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:15.929169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:15.948935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 114/240] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:15.949910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:16.053241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f8a1a-6870-4581-9a41-a4eee68c2997 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:16.082779 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:16.116750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:16.119202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.120443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.120862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.122202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:16.123965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:16.126000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cf31-719b-4451-b95c-a8f87ebe4bd0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:16.195120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:16.195120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.195120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.195120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.195574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:16.195574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:16.195574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:16.195574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 115/241] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:16.196047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:16.300859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a0a61-36d9-48f7-a160-45d04e7537d0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:16.312856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 127/242] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:15 2022] POST /identity/v3/users => generated 293 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:03:16.348933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eada2be-4fbe-4bd6-8116-2629210bcf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:16.349904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada2be-4fbe-4bd6-8116-2629210bcf1e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.350325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada2be-4fbe-4bd6-8116-2629210bcf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.350737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eada2be-4fbe-4bd6-8116-2629210bcf1e None admin] PATH_INFO: `/v3/users/5956f25016f745d08c0cb97a4e8928b9` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.363598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:16.372847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eada2be-4fbe-4bd6-8116-2629210bcf1e None admin] RBAC: Authorizing `identity:get_user(user_id=5956f25016f745d08c0cb97a4e8928b9)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:16.374983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eada2be-4fbe-4bd6-8116-2629210bcf1e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:16.377171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 116/243] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:16 2022] GET /identity/v3/users/5956f25016f745d08c0cb97a4e8928b9 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:16.383282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:16.476053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:16.497718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82756ba7-e577-487b-b5a6-4746b1113e57 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:16.534164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e121d9-8d2b-4610-aa53-15988288c301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:16.535058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e121d9-8d2b-4610-aa53-15988288c301 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.535483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e121d9-8d2b-4610-aa53-15988288c301 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.536080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e121d9-8d2b-4610-aa53-15988288c301 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.608672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058679b-e07e-407b-a40d-23c4fd88313a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.609129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058679b-e07e-407b-a40d-23c4fd88313a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.609852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058679b-e07e-407b-a40d-23c4fd88313a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.610959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 117/244] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:16.613733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e121d9-8d2b-4610-aa53-15988288c301 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:16.615223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e121d9-8d2b-4610-aa53-15988288c301 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:16.617870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2ad8d-ec89-4243-b3f1-398840d88fa7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.618428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2ad8d-ec89-4243-b3f1-398840d88fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.618890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2ad8d-ec89-4243-b3f1-398840d88fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.641220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1ed2ad8d-ec89-4243-b3f1-398840d88fa7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:16.642737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1ed2ad8d-ec89-4243-b3f1-398840d88fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:16.708325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:16.724120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 118/245] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:16.732114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc187738-b4af-4edf-ab93-f7084d17b11a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.732265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 128/246] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:16.732942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc187738-b4af-4edf-ab93-f7084d17b11a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.733427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc187738-b4af-4edf-ab93-f7084d17b11a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.735361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 119/247] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:16.740208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc187738-b4af-4edf-ab93-f7084d17b11a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:16.740596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.740882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.741154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.761441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:16.762640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:16.855355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 129/248] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:16.856251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:16.890096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc187738-b4af-4edf-ab93-f7084d17b11a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:16.923543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:16.933791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:16.934755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:16.935046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:16.938972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:16.938972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:16.939144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:16.940840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 120/249] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:16 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:16.944792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:16.982783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc59b5-ec06-4573-be16-89a6b6dba558 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:16.996601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:17.019779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.020449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.020780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.021052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.033264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:17.038200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:17.039123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:17.040101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 130/250] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:17.040767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.071771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603e07c3-a5b2-4723-85b9-a3f9280cebb9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.136246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.137123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.137545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.138080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.147566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c878f20-6d6b-407b-8f76-71401e4d6c8e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.158951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:17.165426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:17.166467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:17.167434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:17.173613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 121/251] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:17.174448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.193710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.194325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.194600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.194870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.196424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] RBAC: Adding query filter params (role.id=fa67919b5894446a8db1460851778f9d, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=5956f25016f745d08c0cb97a4e8928b9) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:17.196739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:17.198656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b27de-38ae-4837-a060-419505a1c85d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:17.208044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 131/252] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/role_assignments?user.id=5956f25016f745d08c0cb97a4e8928b9&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=fa67919b5894446a8db1460851778f9d => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:17.208044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc5930-ecba-42ba-a815-a8e23d773865 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.208303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc5930-ecba-42ba-a815-a8e23d773865 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.208371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc5930-ecba-42ba-a815-a8e23d773865 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.209860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 132/253] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:17.210589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc5930-ecba-42ba-a815-a8e23d773865 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.263446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:17.265608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:17.346767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc5930-ecba-42ba-a815-a8e23d773865 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.363247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52b597-8a6f-48f7-acf5-a8f26a37ba7b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.402556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.403443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.404383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.404878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.405745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:17.406402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:17.408020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d520f2c-346d-4191-9fc3-d771a98be3f6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:17.418098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 133/254] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:17.420194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.420702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.421170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.442384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:17.444427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:17.452047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.457431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.457926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.458415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.459287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:17.464426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:17.465829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:17.486283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 122/255] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:17.487337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.536396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 134/256] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:17.537954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.613313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:17.649751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad4be7-923c-4bbb-aded-ef0430a8fcfa None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.667837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:17.685770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.686399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.686725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.686994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] PATH_INFO: `/v3/users/29636b2bb2f844d3af8818327af56c4a` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.688159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] RBAC: Authorizing `identity:get_user(user_id=29636b2bb2f844d3af8818327af56c4a)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:17.689584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:17.691321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 123/257] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/users/29636b2bb2f844d3af8818327af56c4a => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:17.691915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.793806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dff70a-0ed1-4d8a-b53b-a6799903ae7b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.828773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.829761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.830135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.830543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.841424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:17.841869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7772ef-9ef1-45b3-ae5f-d18c646372ca None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:17.847896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:17.848963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:17.849927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:17.852301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 124/258] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:17.853105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:17.908362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:17.959005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a00a1a-6da3-4d69-bedb-396ab3996557 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:17.973249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:17.973900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:17.974176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:17.974444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] PATH_INFO: `/v3/users/ef7609cee46046e18efe543352c08ba6` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:17.975573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] RBAC: Authorizing `identity:get_user(user_id=ef7609cee46046e18efe543352c08ba6)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:17.978154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:17.979757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 135/259] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/users/ef7609cee46046e18efe543352c08ba6 => generated 297 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:17.980552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.004740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:18.005517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:18.005891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:18.006257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:18.122326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:18.123983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:18.179014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:18.192555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd631b6-0bb7-4e75-94bb-82ea513ea6bf None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:18.207063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:18.273244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:18.275202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:18.275202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:18.275202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:18.276150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:18.276545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:18.278009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:18.282116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 125/260] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:18.282890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.299315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 136/261] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:18.300314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.409166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ca83a-d148-413d-8d02-0e16eb48633d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:18.478137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:18.478786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:18.479032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:18.479303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:18.482224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:18.483651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:18.485299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 126/262] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:18 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:18.486054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.514637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:18.529858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f7bb15-be0f-4f18-9ca6-5b05db323be7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:18.593745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:18.594398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:18.594629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:18.594891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:18.596183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:18.597625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:18.599412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 137/263] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:18 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:18.599964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.658872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:18.694002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f4977-814a-4815-af52-d5812d20d33c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:18.740385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ff3ef-2511-4887-a003-0a5bb71dae20 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:18.778216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:18.778890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:18.779168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:18.779444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608/users/ef7609cee46046e18efe543352c08ba6/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:18.781593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0187a24b848949bcd600864be9608, user_id=ef7609cee46046e18efe543352c08ba6, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:18.783059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:18.795665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:18.796498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:18.798930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:18.798930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] PATH_INFO: `/v3/users/29636b2bb2f844d3af8818327af56c4a` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:18.798930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] RBAC: Authorizing `identity:get_user(user_id=29636b2bb2f844d3af8818327af56c4a)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:18.799771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:18.801580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 127/264] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:18 2022] GET /identity/v3/users/29636b2bb2f844d3af8818327af56c4a => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:18.802265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.859784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 138/265] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:18 2022] PUT /identity/v3/projects/bcc0187a24b848949bcd600864be9608/users/ef7609cee46046e18efe543352c08ba6/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:18.860681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:18.952359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:18.982016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6e3f1-29cf-4e13-a8b0-8d132c7aeddc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:19.056409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:19.058533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:19.059162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.059444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.059720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.060487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=29636b2bb2f844d3af8818327af56c4a) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:19.060799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:19.062237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ef015-d848-4a6e-8a8a-6116d3a896dd None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:19.076548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 128/266] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:18 2022] GET /identity/v3/role_assignments?user.id=29636b2bb2f844d3af8818327af56c4a&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:19.083831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83538cd2-8138-4fe4-af95-f5c34449916f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:19.136810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9fb22d-a7e3-44eb-88b7-43158af78fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:19.137446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb22d-a7e3-44eb-88b7-43158af78fca None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.137728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb22d-a7e3-44eb-88b7-43158af78fca None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.138001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9fb22d-a7e3-44eb-88b7-43158af78fca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.229221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fb22d-a7e3-44eb-88b7-43158af78fca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:19.230638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9fb22d-a7e3-44eb-88b7-43158af78fca None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:19.321970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fbb58-514e-4a15-980f-23b427336a89 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.322316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fbb58-514e-4a15-980f-23b427336a89 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.322552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fbb58-514e-4a15-980f-23b427336a89 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.323896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 129/267] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:19.329550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eae3d-d0e5-4176-9233-c67e1fdc8a2d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.329829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eae3d-d0e5-4176-9233-c67e1fdc8a2d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.330101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eae3d-d0e5-4176-9233-c67e1fdc8a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.352943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:19.357573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a30eae3d-d0e5-4176-9233-c67e1fdc8a2d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:19.359008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a30eae3d-d0e5-4176-9233-c67e1fdc8a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:19.375095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 139/268] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:19.375095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651369-7cf7-47cd-88b5-ecc80acc1891 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.375095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651369-7cf7-47cd-88b5-ecc80acc1891 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.375095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6651369-7cf7-47cd-88b5-ecc80acc1891 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.375430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 140/269] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:19.383100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651369-7cf7-47cd-88b5-ecc80acc1891 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:19.455104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 130/270] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:19.456729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.457219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.457685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.485740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:19.488683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:19.532018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6651369-7cf7-47cd-88b5-ecc80acc1891 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:19.567158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 131/271] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:19.568021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:19.576637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb632ce8-89dd-4caf-8b76-33ba17243369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:19.577808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb632ce8-89dd-4caf-8b76-33ba17243369 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.578293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb632ce8-89dd-4caf-8b76-33ba17243369 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.578759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb632ce8-89dd-4caf-8b76-33ba17243369 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.580182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb632ce8-89dd-4caf-8b76-33ba17243369 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:19.580606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:19.582460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb632ce8-89dd-4caf-8b76-33ba17243369 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:19.584696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 141/272] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:19 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:19.586780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eb4a8-ed16-41cd-8451-2e2dfbf5ab41 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.587288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eb4a8-ed16-41cd-8451-2e2dfbf5ab41 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.587754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eb4a8-ed16-41cd-8451-2e2dfbf5ab41 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.589382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 142/273] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:19.597019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eb4a8-ed16-41cd-8451-2e2dfbf5ab41 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:19.651274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:19.687342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf29a9c-3ee1-4f3d-b13d-342c48577f58 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:19.700487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eb4a8-ed16-41cd-8451-2e2dfbf5ab41 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:19.725817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9441b13-53bb-4fe0-93d2-6a13f54437f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:19.726523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9441b13-53bb-4fe0-93d2-6a13f54437f7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.727377 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9441b13-53bb-4fe0-93d2-6a13f54437f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.727807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9441b13-53bb-4fe0-93d2-6a13f54437f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.738901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:19.739037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9441b13-53bb-4fe0-93d2-6a13f54437f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:19.739419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.739688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.739951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.740684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] RBAC: Adding query filter params (role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=29636b2bb2f844d3af8818327af56c4a) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:19.740986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:19.742915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-e9441b13-53bb-4fe0-93d2-6a13f54437f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:19.743704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:19.744580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:19.745362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 132/274] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:19.746482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.746482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.746723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.753690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 143/275] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:19 2022] GET /identity/v3/role_assignments?user.id=29636b2bb2f844d3af8818327af56c4a&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=22011eb257084dc58e55ad842a3a3292 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:19.761034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:19.766389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:19.769326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:19.856383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:19.865653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 133/276] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:19.866819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:19.920514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b2f0e-f6bd-4510-9c64-0dc5cfd4bdaa None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:19.938938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:19.959942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:19.961502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:19.962497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:19.962771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:19.963909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:19.965544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:19.967145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 144/277] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:19.967857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:19.996211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79caf-f400-48d8-a844-5be75431dd59 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.044486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.045096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.045372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.045641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.046300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:20.046601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.047968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.057304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 134/278] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:20.061926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.093197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32336348-fee1-4766-8ab9-9c55d016bb2f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.125324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:20.127671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.128334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.128614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.128880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.140008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:20.145132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:20.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 145/279] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:20.153127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.175906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb55c53-6e4c-416b-a125-05ea12fba81e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.200212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:20.215281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.215988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.216346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.216609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.217198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.218782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.253360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df47bf-d271-4b4d-8b9f-d07d3e8c64bf None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.287323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.287948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.288268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.288536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] PATH_INFO: `/v3/users/9c644a8785864cf4a75ae0da204c4e16` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.302302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] RBAC: Authorizing `identity:get_user(user_id=9c644a8785864cf4a75ae0da204c4e16)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.303742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.306135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 146/280] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:20 2022] GET /identity/v3/users/9c644a8785864cf4a75ae0da204c4e16 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:20.307734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.367728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 135/281] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:20 2022] POST /identity/v3/users => generated 293 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:20.367728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.436816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970e435-d5b5-46ac-945d-00a69b0bbc08 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.440309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:20.488029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.488945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.489218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.489750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.490861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:20.491314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.493967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.494523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b4e86-485d-4854-a25d-26f86fdb35f4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.505946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 147/282] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:20.512277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.515038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:20.543314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.543923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.544232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.544500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.545223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, user.id=9c644a8785864cf4a75ae0da204c4e16, scope.system=all) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:20.545530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.546953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.555767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 136/283] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 10 11:03:20 2022] GET /identity/v3/role_assignments?user.id=9c644a8785864cf4a75ae0da204c4e16&scope.system=all&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:20.639335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef615-52f7-4242-b5bb-6a7fbca147d2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.678334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.678998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.679574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.682077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] PATH_INFO: `/v3/users/5956f25016f745d08c0cb97a4e8928b9` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.682757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] RBAC: Authorizing `identity:get_user(user_id=5956f25016f745d08c0cb97a4e8928b9)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.684399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.687029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 148/284] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:20 2022] GET /identity/v3/users/5956f25016f745d08c0cb97a4e8928b9 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:20.692803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.812522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3295c0-6f46-4285-a583-8e1482b170ed None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:20.849403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73a435-03d1-41a4-a81f-35622042fc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:20.850449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a435-03d1-41a4-a81f-35622042fc7c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:20.850920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a435-03d1-41a4-a81f-35622042fc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:20.851406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73a435-03d1-41a4-a81f-35622042fc7c None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:20.853820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73a435-03d1-41a4-a81f-35622042fc7c None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:20.855481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73a435-03d1-41a4-a81f-35622042fc7c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:20.857784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 137/285] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:20 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:20.863298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:20.905955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:20.981908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd4edd-7287-4dae-8502-d669451832a1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:21.032557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba97abe-31f6-4350-b90a-75b528f52e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:21.033241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba97abe-31f6-4350-b90a-75b528f52e72 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:21.033513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba97abe-31f6-4350-b90a-75b528f52e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:21.033780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba97abe-31f6-4350-b90a-75b528f52e72 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608/users/5956f25016f745d08c0cb97a4e8928b9/roles/fa67919b5894446a8db1460851778f9d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:21.035685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba97abe-31f6-4350-b90a-75b528f52e72 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0187a24b848949bcd600864be9608, user_id=5956f25016f745d08c0cb97a4e8928b9, role_id=fa67919b5894446a8db1460851778f9d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:21.037157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba97abe-31f6-4350-b90a-75b528f52e72 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:21.111338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 149/286] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:20 2022] PUT /identity/v3/projects/bcc0187a24b848949bcd600864be9608/users/5956f25016f745d08c0cb97a4e8928b9/roles/fa67919b5894446a8db1460851778f9d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:03:21.264697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d05fb-ea82-42df-b760-cb0008db050f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:21.265035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d05fb-ea82-42df-b760-cb0008db050f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:21.265253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d05fb-ea82-42df-b760-cb0008db050f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:21.266798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 138/287] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:21.272109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:21.272425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:21.272668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:21.292226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:21.293418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:21.364659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 150/288] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:21.401537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d05fb-ea82-42df-b760-cb0008db050f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:21.455824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:21.514983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d05fb-ea82-42df-b760-cb0008db050f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:21.550922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:21.551561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:21.551840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:21.552138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:21.565232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:21.570245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:21.571100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:21.572025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 139/289] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:21.578714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:21.608366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:21.701687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5f08f-ec29-40d4-a4da-af39ea36aaa6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:21.748722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:21.749391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:21.749667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:21.749935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:21.750598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:21.750949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:21.752349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:21.762515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 151/290] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:21.768154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:21.880534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47316594-4918-4330-ba56-56d7299aab74 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:21.888414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:21.917237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:21.917871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:21.918148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:21.918462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] PATH_INFO: `/v3/users/ef7609cee46046e18efe543352c08ba6` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:21.919650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7609cee46046e18efe543352c08ba6)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:21.921356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:21.923466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 140/291] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:21 2022] GET /identity/v3/users/ef7609cee46046e18efe543352c08ba6 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:21.928924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:22.040147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:22.059463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084bc675-5751-4495-8287-48a2748b8700 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:22.093799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:22.094415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:22.094698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:22.094975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:22.174082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:22.175543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:22.317881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:22.344419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 152/292] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:22.345780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:22.510269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df337f-21d1-4c1f-8f02-7c84fea93f46 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:22.544850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f9940-3a69-432a-9608-5db7c64f3f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:22.545590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9940-3a69-432a-9608-5db7c64f3f4d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:22.545812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9940-3a69-432a-9608-5db7c64f3f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:22.546087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9940-3a69-432a-9608-5db7c64f3f4d None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:22.547324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f9940-3a69-432a-9608-5db7c64f3f4d None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:22.548814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f9940-3a69-432a-9608-5db7c64f3f4d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:22.550452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 141/293] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:22 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:22.561097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:22.683663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c30a2b-513c-4cf4-80e9-e493a3a449b6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:22.718840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:22.719472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:22.719744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:22.719975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:22.720720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=ef7609cee46046e18efe543352c08ba6) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:22.721027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:22.722478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42025e50-7897-4f58-8c68-4256d1438bac None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:22.729076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 153/294] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:22 2022] GET /identity/v3/role_assignments?user.id=ef7609cee46046e18efe543352c08ba6&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:23.013443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57f9c2-4383-4d66-a0ce-b5310319b35e None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.013632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57f9c2-4383-4d66-a0ce-b5310319b35e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.013891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57f9c2-4383-4d66-a0ce-b5310319b35e None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.015223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 142/295] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:23.020301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5932f-ac06-4cd0-b737-a501487c5e01 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.020591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5932f-ac06-4cd0-b737-a501487c5e01 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.020871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5932f-ac06-4cd0-b737-a501487c5e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.040616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-bbb5932f-ac06-4cd0-b737-a501487c5e01 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:23.041806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bbb5932f-ac06-4cd0-b737-a501487c5e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:23.095871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.118160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 154/296] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:23.142111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57f9c2-4383-4d66-a0ce-b5310319b35e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.154657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda38f47-7b36-4e2a-bfa6-b5734275ebe9 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.162506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda38f47-7b36-4e2a-bfa6-b5734275ebe9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.162506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda38f47-7b36-4e2a-bfa6-b5734275ebe9 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.162506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 155/297] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:23.165853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.166137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.166430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.186238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:23.187426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:23.198363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.268292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 156/298] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:23.270298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57f9c2-4383-4d66-a0ce-b5310319b35e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.282346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.312029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.312681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.312961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.313232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.326450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:23.331871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:23.332973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:23.334106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 143/299] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:23.339918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.404563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692be3c5-a726-417d-af94-da8cabad4fef None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.419593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.444579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2c333-2f37-4c31-8dc1-0f8669c7dc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.448105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c333-2f37-4c31-8dc1-0f8669c7dc18 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.448105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c333-2f37-4c31-8dc1-0f8669c7dc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.448105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c333-2f37-4c31-8dc1-0f8669c7dc18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.453382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60002d8b-b603-44eb-83b3-151b69b50c2e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.455065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.456726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef2c333-2f37-4c31-8dc1-0f8669c7dc18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-9ef2c333-2f37-4c31-8dc1-0f8669c7dc18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:23.462207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:23.463079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:23.463907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 157/300] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:23.465149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc024-39db-44b0-91b4-682ac4de2a66 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.465149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc024-39db-44b0-91b4-682ac4de2a66 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.465400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fc024-39db-44b0-91b4-682ac4de2a66 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.466612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 158/301] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:23.468849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc024-39db-44b0-91b4-682ac4de2a66 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.493493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.494147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.494434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.494711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.495453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:23.495809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:23.497393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dd0cd-2198-42e0-84f7-875881354f8c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:23.510980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 144/302] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:23.510980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.510980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.510980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.528214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:23.529340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:23.578341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fc024-39db-44b0-91b4-682ac4de2a66 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.616024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 145/303] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:23.616024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.626270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.627013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.627354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.627689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.628484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:23.628848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:23.630286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:23.639902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 159/304] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:23.640743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.672410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.715373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.722685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7ca52-1ad8-485d-a2db-b47213a8f19a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.747630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d353a-fb39-4cf1-80c0-c273d96828d5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.760911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.761626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.761975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.762251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] PATH_INFO: `/v3/users/bf4f1f7d86d845e4b996a0daa732e7e5` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.777351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4f1f7d86d845e4b996a0daa732e7e5)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:23.778716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:23.780357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 146/305] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/users/bf4f1f7d86d845e4b996a0daa732e7e5 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:23.785220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.793226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.793937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.794285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.794623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.795785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:23.797343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:23.799100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 160/306] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:23.799779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.892777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9e833-b43e-4e9a-aa8f-39d7921945e2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.896643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:23.900211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eafff19-17df-4965-a72d-4fe7ca9e9def None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:23.925726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.926352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.926630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.926901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] PATH_INFO: `/v3/users/9c644a8785864cf4a75ae0da204c4e16` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.928065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] RBAC: Authorizing `identity:get_user(user_id=9c644a8785864cf4a75ae0da204c4e16)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:23.929829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:23.931489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 147/307] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/users/9c644a8785864cf4a75ae0da204c4e16 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:23.932253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.932824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:23.933800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.934239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.934835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.935563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] RBAC: Adding query filter params (role.id=fa67919b5894446a8db1460851778f9d, user.id=bf4f1f7d86d845e4b996a0daa732e7e5, scope.system=all) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:23.935870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:23.937340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ff68e-6f14-4944-a0e2-a1fbe9594347 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:23.942348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 161/308] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 10 11:03:23 2022] GET /identity/v3/role_assignments?user.id=bf4f1f7d86d845e4b996a0daa732e7e5&scope.system=all&role.id=fa67919b5894446a8db1460851778f9d => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:23.944582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7146-2e61-483f-9fe5-e08265dd338b None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:23.944867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7146-2e61-483f-9fe5-e08265dd338b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:23.945665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca7146-2e61-483f-9fe5-e08265dd338b None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:23.946967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 162/309] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:23.947688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca7146-2e61-483f-9fe5-e08265dd338b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:23.961405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:24.051538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aeaa58-6c85-4907-9b54-ecac738a82c4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:24.061508 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca7146-2e61-483f-9fe5-e08265dd338b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:24.095112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24573d2-567c-4e81-9708-72239563ac38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:24.095734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24573d2-567c-4e81-9708-72239563ac38 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:24.096049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24573d2-567c-4e81-9708-72239563ac38 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:24.096332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24573d2-567c-4e81-9708-72239563ac38 None admin] PATH_INFO: `/v3/groups` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:24.096968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24573d2-567c-4e81-9708-72239563ac38 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:24.098417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24573d2-567c-4e81-9708-72239563ac38 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:24.102539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:24.103191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:24.103464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:24.103732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] PATH_INFO: `/v3/system/users/9c644a8785864cf4a75ae0da204c4e16/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:24.105165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9c644a8785864cf4a75ae0da204c4e16, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:24.106530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:24.207923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 148/310] 10.222.0.13 () {60 vars in 1134 bytes} [Wed Aug 10 11:03:23 2022] POST /identity/v3/groups => generated 229 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:24.209265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:24.209549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:24.209817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:24.231350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:24.232612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:24.256157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 163/311] 10.222.0.13 () {58 vars in 1334 bytes} [Wed Aug 10 11:03:23 2022] PUT /identity/v3/system/users/9c644a8785864cf4a75ae0da204c4e16/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:24.327175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 149/312] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:24.350281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:24.389093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:24.475891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6123d73-33da-4948-9822-8f9acf7e33b2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:24.512418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:24.513052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:24.513327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:24.513593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:24.524050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:24.528920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:24.529807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:24.530595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 164/313] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:24.539313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:24.551063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:24.649448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bffda7-0689-4ee5-8996-5e68f718b985 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:24.689040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:24.689642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:24.689916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:24.690183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:24.690842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:24.691138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:24.692542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:24.701849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 150/314] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:24.707216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:24.803772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:24.816176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983fdeba-6cab-4786-9d9b-f38c07ca7f46 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:24.851045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:24.857091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:24.857476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:24.857856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] PATH_INFO: `/v3/users/5956f25016f745d08c0cb97a4e8928b9` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:24.859142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] RBAC: Authorizing `identity:get_user(user_id=5956f25016f745d08c0cb97a4e8928b9)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:24.861236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:24.863156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 165/315] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:24 2022] GET /identity/v3/users/5956f25016f745d08c0cb97a4e8928b9 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:24.868696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:24.943553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:24.977371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e067bd-084f-4127-bcbc-1c2c31bf2e31 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:25.013096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:25.013704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:25.013978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:25.014244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:25.087421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:25.088965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:25.164830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:25.196796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 151/316] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:25.202796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:25.308398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda1c66-35b4-466c-91a5-295c1c7083c1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:25.343251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998ef7d4-2f62-4714-a15f-d092feea5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:25.343872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ef7d4-2f62-4714-a15f-d092feea5485 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:25.344274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ef7d4-2f62-4714-a15f-d092feea5485 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:25.344543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998ef7d4-2f62-4714-a15f-d092feea5485 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:25.345658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ef7d4-2f62-4714-a15f-d092feea5485 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:25.347095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998ef7d4-2f62-4714-a15f-d092feea5485 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:25.349064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 166/317] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:25 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:25.354791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:25.478880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87072493-b84d-4cf2-a085-040f83116327 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:25.515332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:25.515941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:25.516250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:25.516535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:25.517249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] RBAC: Adding query filter params (role.id=fa67919b5894446a8db1460851778f9d, scope.project.id=bcc0187a24b848949bcd600864be9608, user.id=5956f25016f745d08c0cb97a4e8928b9) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:25.517557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:25.519012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54df658-7652-41e1-8489-f2e4b3ca7c87 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:25.526473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 152/318] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:25 2022] GET /identity/v3/role_assignments?user.id=5956f25016f745d08c0cb97a4e8928b9&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=fa67919b5894446a8db1460851778f9d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:26.041047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17ad18-a224-4c63-bc0a-44dff4d4049f None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.041595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17ad18-a224-4c63-bc0a-44dff4d4049f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.041595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17ad18-a224-4c63-bc0a-44dff4d4049f None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.043039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 167/319] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:26.048172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.048567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.048766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.067090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:26.068297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:26.097260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.137421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 153/320] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:26.158656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17ad18-a224-4c63-bc0a-44dff4d4049f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.185677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.261237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17ad18-a224-4c63-bc0a-44dff4d4049f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.295538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc314f-7e32-4a8e-b48e-3885c5f15a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.296092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc314f-7e32-4a8e-b48e-3885c5f15a07 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.296369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc314f-7e32-4a8e-b48e-3885c5f15a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.296634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc314f-7e32-4a8e-b48e-3885c5f15a07 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.307534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc314f-7e32-4a8e-b48e-3885c5f15a07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-62fc314f-7e32-4a8e-b48e-3885c5f15a07 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:26.312225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:26.313099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:26.313914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:26.316079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 168/321] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:26.319380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.345399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a4666-1b42-4f7b-b2ce-2459e95b2618 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.345709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a4666-1b42-4f7b-b2ce-2459e95b2618 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.345957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a4666-1b42-4f7b-b2ce-2459e95b2618 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.347347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 169/322] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:26.352364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fbd9e-7ac7-4cac-bdbe-8cf1c82234a0 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.352642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fbd9e-7ac7-4cac-bdbe-8cf1c82234a0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.352920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964fbd9e-7ac7-4cac-bdbe-8cf1c82234a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.371190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-964fbd9e-7ac7-4cac-bdbe-8cf1c82234a0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:26.372579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-964fbd9e-7ac7-4cac-bdbe-8cf1c82234a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:26.429111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b9f31-ea2e-44ac-9ef5-5600bb165f30 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.447301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 170/323] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:26.448595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b074-dfd6-4e2d-95cd-604a462bf090 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.448890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b074-dfd6-4e2d-95cd-604a462bf090 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.449169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2637b074-dfd6-4e2d-95cd-604a462bf090 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.450398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 171/324] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:26.457305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.457583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.457863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.466936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.467541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.467822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.468120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.468784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:26.469083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:26.470450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:26.475822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:26.477860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:26.479143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 154/325] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:26.479927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.487573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.512755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.550604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 172/326] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:26.551375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.583720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c468e1-2465-49c0-8e8c-838f069a71cf None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.617461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.618077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.618350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.618616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.628288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:26.633659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:26.634700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:26.635526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 155/327] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:26.636225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.655430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165b29d-6e57-4aef-8b08-92257af30094 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.688496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.689114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.689386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.689653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] PATH_INFO: `/v3/users/bf4f1f7d86d845e4b996a0daa732e7e5` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.690747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4f1f7d86d845e4b996a0daa732e7e5)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:26.692200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:26.693758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 173/328] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/users/bf4f1f7d86d845e4b996a0daa732e7e5 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:26.699082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.709065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.721884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.733604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8af712-dd72-4d44-af37-6eb669f830fd None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.767045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.767665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.767946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.768245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.777630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:26.777630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:26.777630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 156/329] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:26.777630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.795065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae61c1e-c5e8-422a-adbc-ce39bf788bf9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.828776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.829371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.829646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.829911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.830551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:26.830854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:26.832234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:26.841436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 174/330] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:26.844461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:26.872013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb6e95-b751-4915-86a3-6b4b520c3435 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.906085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.906720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.906995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.907263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] PATH_INFO: `/v3/system/users/bf4f1f7d86d845e4b996a0daa732e7e5/roles/fa67919b5894446a8db1460851778f9d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.908667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bf4f1f7d86d845e4b996a0daa732e7e5, role_id=fa67919b5894446a8db1460851778f9d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:26.910023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:26.918348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:26.941750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a8fca-f570-46d3-b6bf-f9aa5c6f9a13 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:26.973671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 157/331] 10.222.0.13 () {58 vars in 1334 bytes} [Wed Aug 10 11:03:26 2022] PUT /identity/v3/system/users/bf4f1f7d86d845e4b996a0daa732e7e5/roles/fa67919b5894446a8db1460851778f9d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:26.974537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:26.975140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:26.975418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:26.975690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] PATH_INFO: `/v3/groups` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:26.976363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:26.977814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:26.986107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:27.005588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:27.069053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 175/332] 10.222.0.13 () {60 vars in 1134 bytes} [Wed Aug 10 11:03:26 2022] POST /identity/v3/groups => generated 226 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:27.099949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d38e2-edd7-460e-a1a1-13b6344d4b76 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:27.143766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d1748b-788f-42e0-b76d-d33e24ca9885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:27.144448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1748b-788f-42e0-b76d-d33e24ca9885 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:27.144723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1748b-788f-42e0-b76d-d33e24ca9885 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:27.144989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d1748b-788f-42e0-b76d-d33e24ca9885 None admin] PATH_INFO: `/v3/users/9c644a8785864cf4a75ae0da204c4e16` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:27.146216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1748b-788f-42e0-b76d-d33e24ca9885 None admin] RBAC: Authorizing `identity:get_user(user_id=9c644a8785864cf4a75ae0da204c4e16)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:27.147675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d1748b-788f-42e0-b76d-d33e24ca9885 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:27.149880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 158/333] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:26 2022] GET /identity/v3/users/9c644a8785864cf4a75ae0da204c4e16 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:27.158187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:27.268413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf55eb-7be7-4343-8db9-ff0b3890c90d None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:27.305787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:27.306476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:27.306751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:27.307021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:27.307744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] RBAC: Adding query filter params (role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, user.id=9c644a8785864cf4a75ae0da204c4e16, scope.system=all) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:27.308082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:27.309527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbc0a4-4b83-42fa-ba4f-e91631969273 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:27.311356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:27.323816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 176/334] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 10 11:03:27 2022] GET /identity/v3/role_assignments?user.id=9c644a8785864cf4a75ae0da204c4e16&scope.system=all&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:29.245974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae1c6-022e-44c7-b0db-060c8ad3a12d None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:29.246932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae1c6-022e-44c7-b0db-060c8ad3a12d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:29.247378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ae1c6-022e-44c7-b0db-060c8ad3a12d None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:29.249269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 159/335] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:29.254904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3d2d-983d-4cbb-9b40-1e6e6a85641f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:29.255368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3d2d-983d-4cbb-9b40-1e6e6a85641f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:29.255554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3d2d-983d-4cbb-9b40-1e6e6a85641f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:29.285029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8d2e3d2d-983d-4cbb-9b40-1e6e6a85641f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:29.286304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8d2e3d2d-983d-4cbb-9b40-1e6e6a85641f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:29.379187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 177/336] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:29.439549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae1c6-022e-44c7-b0db-060c8ad3a12d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:29.517919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb310b-9225-4110-a636-05939242ab90 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:29.518100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb310b-9225-4110-a636-05939242ab90 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:29.518439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb310b-9225-4110-a636-05939242ab90 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:29.519916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 178/337] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:29.525730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:29.526011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:29.526284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:29.555419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:29.565007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:29.566204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:29.576681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ae1c6-022e-44c7-b0db-060c8ad3a12d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:29.646397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:29.647035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:29.647310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:29.647572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:29.676682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:29.682889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:29.683751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:29.684970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 160/338] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:29.698334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:29.732525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 179/339] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 10 11:03:29.764296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:29.872471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:29.909932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf46c35-7814-4e40-9dd5-45319f522d32 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:29.956318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24147836-a3d8-47ba-9882-6373248ee951 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:29.974955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:29.975594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:29.975878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:29.976180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:29.976957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] RBAC: Adding query filter params (name=reader) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:29.977260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:29.978660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd80c-3333-47df-9971-66965eb222de None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:30.001983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 161/340] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:30.003257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d653b9-8e87-4f65-bf4f-8b47c8f47c63 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.003544 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d653b9-8e87-4f65-bf4f-8b47c8f47c63 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.003820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d653b9-8e87-4f65-bf4f-8b47c8f47c63 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.006889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 162/341] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:30.007694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d653b9-8e87-4f65-bf4f-8b47c8f47c63 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.019028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cecc569-2c77-47b2-9cbe-a71cbd290296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.019739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc569-2c77-47b2-9cbe-a71cbd290296 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.020044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc569-2c77-47b2-9cbe-a71cbd290296 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.020323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cecc569-2c77-47b2-9cbe-a71cbd290296 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.028467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.037756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cecc569-2c77-47b2-9cbe-a71cbd290296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-4cecc569-2c77-47b2-9cbe-a71cbd290296 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:30.043844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:30.044889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:30.045679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:30.048270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 180/342] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:30.050175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fe98e-9f86-48ee-bb0d-5809800b091b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.050783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fe98e-9f86-48ee-bb0d-5809800b091b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.051363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fe98e-9f86-48ee-bb0d-5809800b091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.078891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-874fe98e-9f86-48ee-bb0d-5809800b091b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:30.081257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-874fe98e-9f86-48ee-bb0d-5809800b091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:30.133206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d653b9-8e87-4f65-bf4f-8b47c8f47c63 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.163297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 181/343] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:30.165882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d00aec-5222-4ffd-a8f7-89843aa0d25d None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.166355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d00aec-5222-4ffd-a8f7-89843aa0d25d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.166930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d00aec-5222-4ffd-a8f7-89843aa0d25d None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.168931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 182/344] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:30.169933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d00aec-5222-4ffd-a8f7-89843aa0d25d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.182045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f704605-4af1-4832-a7ee-831c8dc00f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.182738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f704605-4af1-4832-a7ee-831c8dc00f53 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.184752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f704605-4af1-4832-a7ee-831c8dc00f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.184752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f704605-4af1-4832-a7ee-831c8dc00f53 None admin] PATH_INFO: `/v3/users/bf4f1f7d86d845e4b996a0daa732e7e5` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.188158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f704605-4af1-4832-a7ee-831c8dc00f53 None admin] RBAC: Authorizing `identity:get_user(user_id=bf4f1f7d86d845e4b996a0daa732e7e5)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:30.188158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f704605-4af1-4832-a7ee-831c8dc00f53 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:30.189028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 163/345] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/users/bf4f1f7d86d845e4b996a0daa732e7e5 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:30.190712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.191147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.191416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.212039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:30.213252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:30.225600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.297916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 164/346] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:30.298784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.307386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d00aec-5222-4ffd-a8f7-89843aa0d25d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.310591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.356481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.357119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.357402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.357673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.359634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:30.359939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:30.361334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:30.379535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 183/347] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:30.386865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.433142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdde871-a378-4cd4-a851-2b603aed6bdf None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.484667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.485361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.486066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.486349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.487385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] RBAC: Adding query filter params (role.id=fa67919b5894446a8db1460851778f9d, user.id=bf4f1f7d86d845e4b996a0daa732e7e5, scope.system=all) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:30.487698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:30.490052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:30.507103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 165/348] 10.222.0.13 () {56 vars in 1308 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/role_assignments?user.id=bf4f1f7d86d845e4b996a0daa732e7e5&scope.system=all&role.id=fa67919b5894446a8db1460851778f9d => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:30.509208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.518058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eceb0-1e04-42d9-8d6a-f431b516e1d7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.526232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.584209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479c92-c917-4167-86c8-b99343050665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.584856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479c92-c917-4167-86c8-b99343050665 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.585135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479c92-c917-4167-86c8-b99343050665 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.585405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35479c92-c917-4167-86c8-b99343050665 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.602349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35479c92-c917-4167-86c8-b99343050665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:30.603666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-35479c92-c917-4167-86c8-b99343050665 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:30.607375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:30.608571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:30.609569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:30.610213 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 184/349] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:30.610995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479c92-c917-4167-86c8-b99343050665 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.658284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1ed0c1-7df8-4cd3-b33b-07bf64c0006d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.707267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.708461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.708887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.709303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.723888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:30.729954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:30.731132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:30.732070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:30.732773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 166/350] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:30.733497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:30.761440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35479c92-c917-4167-86c8-b99343050665 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.810665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.811290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.811569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.811841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.853117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.861008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a38c90-2437-4361-8d4c-56ae1ecb4d6f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:30.889950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:30.899161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:30.900681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:30.900681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:30.900681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:30.901501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:30.901910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:30.903378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:30.905548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:30.907332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:30.912582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 167/351] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:30.913408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.026404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc26b18-c03a-4eed-9f3c-6d7f3b243fec None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.061736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 185/352] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:31.062820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.064977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.065688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.066074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.066352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.067018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:31.067318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.068886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.082002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 168/353] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:31.082955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.140679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:31.143154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:31.191496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327858-de50-4ffd-b405-63d20ae826ce None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.203345 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b40ac1-a72b-404f-97be-28336f99c155 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.232458 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626edf02-d142-4a86-ae85-3fd3ac94860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.233631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626edf02-d142-4a86-ae85-3fd3ac94860e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.233631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626edf02-d142-4a86-ae85-3fd3ac94860e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.233815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626edf02-d142-4a86-ae85-3fd3ac94860e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.246968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.247617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.247897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.248224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.249376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.250812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.252634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 169/354] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:31.253507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.317968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626edf02-d142-4a86-ae85-3fd3ac94860e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.320497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626edf02-d142-4a86-ae85-3fd3ac94860e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.394834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fba15-a3f7-4e79-b0f7-6e77bfbb7fd7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.412575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:31.419602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:31.441190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f6308-db0e-414f-8707-e6b169235bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.441864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f6308-db0e-414f-8707-e6b169235bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.442086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f6308-db0e-414f-8707-e6b169235bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.442355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f6308-db0e-414f-8707-e6b169235bc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.466859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 186/355] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:31.468260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b74bc5-0fc0-4c5a-a350-edda3a30b22e None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.468551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b74bc5-0fc0-4c5a-a350-edda3a30b22e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.468824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b74bc5-0fc0-4c5a-a350-edda3a30b22e None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.474386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 187/356] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:31.474386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b74bc5-0fc0-4c5a-a350-edda3a30b22e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.536118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f6308-db0e-414f-8707-e6b169235bc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.537719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f6308-db0e-414f-8707-e6b169235bc9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.595017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b74bc5-0fc0-4c5a-a350-edda3a30b22e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.645762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.646676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.646958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.647230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.655083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.656289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.658117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 188/357] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:31.658867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.671249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 170/358] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:31.672557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.672840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.673115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.701324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:31.702863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:31.708656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:31.726564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:31.786741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79935997-4ede-4a33-ae11-a292c12c1c36 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.800640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 171/359] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:31.801521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.830007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.830957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.831430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.831903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.833328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.834944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.837751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 189/360] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:31.838374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.916193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc9e15-a652-4541-9a1a-23c8c50d6446 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.948567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eeab3e-9386-4561-b7bf-f037986417ab None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:31.956527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.957076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.957361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.957636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.958402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=d858077827a543fd96cbd471437ac078) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:31.958716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:31.960246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27217b-2032-4ae4-b83f-729ab4bbda39 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:31.966796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 172/361] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=22011eb257084dc58e55ad842a3a3292 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:31.967832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8009b8-73f6-4d30-b51a-5cfcd9d9c69b None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.968282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8009b8-73f6-4d30-b51a-5cfcd9d9c69b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.968559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8009b8-73f6-4d30-b51a-5cfcd9d9c69b None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.971098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 173/362] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:31.971098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8009b8-73f6-4d30-b51a-5cfcd9d9c69b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:31.997111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:31.997111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:31.997418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:31.997539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:31.999856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:32.001441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:32.003203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 190/363] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:32.004640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.044682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.125910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.141636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8009b8-73f6-4d30-b51a-5cfcd9d9c69b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.148845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915c3d9-300b-4770-af06-0dbc1a4c8daf None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.188784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.189586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.189792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.190016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3c118-4a40-4bbd-9c45-a1fce5d42994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.190131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.190651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3c118-4a40-4bbd-9c45-a1fce5d42994 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.190930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3c118-4a40-4bbd-9c45-a1fce5d42994 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.191204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3c118-4a40-4bbd-9c45-a1fce5d42994 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.192341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3c118-4a40-4bbd-9c45-a1fce5d42994 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:32.193775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3c118-4a40-4bbd-9c45-a1fce5d42994 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:32.195397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 191/364] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:32.196829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.197119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.197391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.205707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:32.213339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:32.214278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:32.215143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 174/365] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:32.215902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.218167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:32.218590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:32.250669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.303437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.311254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 192/366] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:32.312223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.347644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b78b0-2a59-474f-8101-85c5fcbfdfad None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.402334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.403100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.403386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.403763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.404950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:32.406532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:32.408508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 175/367] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:32.409315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.453841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39232b80-fa28-4bd8-bfd0-ac2a8c4ffd4c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.496102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.496940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.497516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.497939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.499311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=d858077827a543fd96cbd471437ac078) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:32.499773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:32.501402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:32.509855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 193/368] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:32.510998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.550300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac523ea8-d8b0-4f06-bdfb-f1aaf59917a2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.556638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.595919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.596583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.597215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.597543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.598539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:32.598843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:32.600259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:32.613239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 176/369] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:32.614109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.618904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.648845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e856cae-06b2-476b-8512-38e9e2391370 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.699018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.700869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.701155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.701428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.723552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:32.730970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:32.731869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:32.732901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:32.735171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 194/370] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:32.736205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.766891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987fafe1-e348-4d5e-b4de-b70017907a1d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.822192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.823135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.823671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.824164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.825176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=bcc0187a24b848949bcd600864be9608) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:32.825663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:32.827342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:32.833621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 177/371] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:32.840694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:32.850268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:32.869153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e67514-cb9b-4824-98f9-ed5488243140 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:32.914151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:32.914868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:32.915225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:32.915569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:32.972290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:33.004483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.006797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.016511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc027c-b69d-411b-b131-a2250a821d72 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:33.065258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:33.066135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:33.066514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:33.066859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:33.067781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:33.068174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.070392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.081591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 178/372] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:33.088212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:33.132476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:33.164348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 195/373] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:33.165480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:33.248809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38f5f7-6307-4d94-87fe-3183e74c2522 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:33.302260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:33.320021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:33.320021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:33.320021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:33.320021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:33.342522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746409ea-d813-4665-8c5a-753e9bd5f532 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:33.380819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:33.381464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:33.381742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:33.382015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:33.383824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.385298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.387377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 196/374] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:33 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:33.393962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:33.430912 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.433735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.484240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:33.545065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d28efa-4cdd-4116-812d-8f6084e67e3e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:33.609417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 179/375] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:33.615611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:33.619257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:33.620457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:33.620457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:33.620646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:33.621669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.623101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.636019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 197/376] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:33 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:33.637289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:33.653569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:33.807891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d954b4b-ad35-4dfd-85a7-fd8b262fbb2e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:33.828897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad47c69-3c54-4d44-bef4-6b651fc8992c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:33.875654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:33.877192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:33.877917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:33.878287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:33.878630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] PATH_INFO: `/v3/projects/1f5409bf396e4a7cbe05519266660a63` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:33.880675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] RBAC: Authorizing `identity:get_project(project_id=1f5409bf396e4a7cbe05519266660a63)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.883990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.884859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 180/377] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:33 2022] GET /identity/v3/projects/1f5409bf396e4a7cbe05519266660a63 => generated 344 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:33.890318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:33.893844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:33.894693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:33.895158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:33.895593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:33.896505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=bcc0187a24b848949bcd600864be9608) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:33.896996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:33.899734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:33.906764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 198/378] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:33 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=22011eb257084dc58e55ad842a3a3292 => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:34.036488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:34.046393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3489e75-05d4-478c-9699-4f110d2f22aa None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:34.092108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799e0f4-0383-4d14-a1da-ac7f58ded860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:34.092752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799e0f4-0383-4d14-a1da-ac7f58ded860 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:34.093033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799e0f4-0383-4d14-a1da-ac7f58ded860 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:34.093309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799e0f4-0383-4d14-a1da-ac7f58ded860 None admin] PATH_INFO: `/v3/groups/e7b9aff6911e40ca9b5ba8505f354745` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:34.112831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799e0f4-0383-4d14-a1da-ac7f58ded860 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b9aff6911e40ca9b5ba8505f354745)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:34.114325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799e0f4-0383-4d14-a1da-ac7f58ded860 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:34.116406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 181/379] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:33 2022] GET /identity/v3/groups/e7b9aff6911e40ca9b5ba8505f354745 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:34.122825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:34.239763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395433ce-a19b-49a9-a4c1-5415c9059ad3 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:34.240067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395433ce-a19b-49a9-a4c1-5415c9059ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:34.240296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395433ce-a19b-49a9-a4c1-5415c9059ad3 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:34.243402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 182/380] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:34.247415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:34.247701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:34.247977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:34.281151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:34.282329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:34.314276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c45ffb-3d8f-4dad-9258-6cfd5bf22aa6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:34.393798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:34.394423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:34.394701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:34.394968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:34.395704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] RBAC: Adding query filter params (group.id=e7b9aff6911e40ca9b5ba8505f354745, role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=1f5409bf396e4a7cbe05519266660a63) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:34.396022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:34.397506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:34.412166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 199/381] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:34 2022] GET /identity/v3/role_assignments?group.id=e7b9aff6911e40ca9b5ba8505f354745&scope.project.id=1f5409bf396e4a7cbe05519266660a63&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 260 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:34.426928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 183/382] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:34.461881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:34.536950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:34.599449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ae9ce-e9ac-4d58-ae29-e83dbe5468c5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:34.646613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:34.647294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:34.647538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:34.647812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:34.665526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:34.670884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:34.671856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:34.672679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:34.676330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 200/383] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:34.678938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:34.779125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:34.796778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a31f6-dc43-42e9-8ec0-7db0dfca1156 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:34.839252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:34.839873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:34.840189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:34.840462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:34.841170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:34.841444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:34.842829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:34.852035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 184/384] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:34.857660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:34.973043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab8383-b7a7-4e5e-ae3d-fcee2833cc0a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:34.990010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.026972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6ad75-4741-4347-8382-531874335362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.027616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ad75-4741-4347-8382-531874335362 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.027891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ad75-4741-4347-8382-531874335362 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.028185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6ad75-4741-4347-8382-531874335362 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.029297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6ad75-4741-4347-8382-531874335362 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:35.030742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6ad75-4741-4347-8382-531874335362 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:35.032700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 201/385] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:34 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:35.038438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.180764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee021606-2317-4eca-96b8-d6b01ccd2132 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:35.197525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74049dd5-f1cd-4613-922c-dae5b8663c10 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.197847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74049dd5-f1cd-4613-922c-dae5b8663c10 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.198070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74049dd5-f1cd-4613-922c-dae5b8663c10 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.199392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 202/386] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:35.222401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce532-2c33-413e-9825-8316e0188cbd None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.222766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce532-2c33-413e-9825-8316e0188cbd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.223037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce532-2c33-413e-9825-8316e0188cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.223619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.224283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.224564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.224840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.228713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:35.230149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:35.231810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 185/387] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:35.237863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.256045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-371ce532-2c33-413e-9825-8316e0188cbd None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:35.257239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-371ce532-2c33-413e-9825-8316e0188cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:35.265405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.340357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a1644-0031-4582-a8ed-7e55abcb5ec4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:35.386688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.387378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.387661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.387932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.390185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] RBAC: Authorizing `identity:create_grant(project_id=d858077827a543fd96cbd471437ac078, group_id=c4bcdeb5b770407895618e9438dc8fc8, role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:35.391638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:35.414253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 203/388] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:35.415903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c02b5-eb83-4bea-b0de-765fc4c89849 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.419643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c02b5-eb83-4bea-b0de-765fc4c89849 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.419643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c02b5-eb83-4bea-b0de-765fc4c89849 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.419643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 204/389] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:35.424811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.425189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.425558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.445944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:35.447265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:35.462063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 186/390] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 10 11:03:35 2022] PUT /identity/v3/projects/d858077827a543fd96cbd471437ac078/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/22011eb257084dc58e55ad842a3a3292 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:03:35.468531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.577385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 205/391] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:35.599007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.660539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.665004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.677024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae3c3a-9d60-460d-8b00-676e33459a4f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:35.723586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.724927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.725995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.727372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.762732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9230082-fcc0-4315-89d2-94155b48c48d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:35.768649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:35.773561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:35.774444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:35.775232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 187/392] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:35.781936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.808385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:35.809077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:35.809352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:35.809619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:35.831752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:35.840706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:35.841587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:35.842369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:35.846056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 206/393] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:35.851902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:35.994787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:35.999209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15d79d-298f-4547-a3b7-48a7776787af None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.038514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc33ee2-face-4615-af7f-5f5fcaaa9486 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.049943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.050555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.050827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.051096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.051753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:36.052074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.053450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.062821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 207/394] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:36.068333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.132054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.132699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.132994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.133238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.133906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:36.134204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.135580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.154490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 188/395] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:36.160557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.165320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:36.235813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a363ed-f303-4693-a39b-385292d0d0d6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.276433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc25bc5-c759-4001-8fed-bec328c6f921 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.314304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.314934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.315211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.315478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.317045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.318463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.320501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 189/396] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:36 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:36.325865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.329303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.329964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.330244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.330517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.336884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.338361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.340336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 208/397] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:36 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:36.357058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.409962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:36.444355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef438-007c-4419-88be-5a67d0b73835 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.452343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:36.481347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.481968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.482485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.482898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.484293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.486981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.489169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 190/398] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:36 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:36.495512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.585809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a9e3f-2ab4-4212-bc76-a8ce0d688eff None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.626352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.627640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.628092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.628655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.630120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.631876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.637925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 209/399] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:36 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:36.640728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:36.710844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa6de4-9280-4a5a-a767-33d7cff816b6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.748887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc3d2d-829d-4d9d-9193-6e92ce6484cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.749529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3d2d-829d-4d9d-9193-6e92ce6484cc None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.749813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3d2d-829d-4d9d-9193-6e92ce6484cc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.750097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc3d2d-829d-4d9d-9193-6e92ce6484cc None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.752132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc3d2d-829d-4d9d-9193-6e92ce6484cc None admin] RBAC: Authorizing `identity:create_grant(project_id=d858077827a543fd96cbd471437ac078, group_id=c4bcdeb5b770407895618e9438dc8fc8, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.753679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc3d2d-829d-4d9d-9193-6e92ce6484cc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.765733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:36.771483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f820d0-d001-4745-8802-c954fba35cc4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:36.808453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0592b10-7c04-4e5c-9a97-b07cf0da7c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:36.811950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592b10-7c04-4e5c-9a97-b07cf0da7c6a None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.811950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592b10-7c04-4e5c-9a97-b07cf0da7c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.811950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0592b10-7c04-4e5c-9a97-b07cf0da7c6a None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.811950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0592b10-7c04-4e5c-9a97-b07cf0da7c6a None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0187a24b848949bcd600864be9608, group_id=c4bcdeb5b770407895618e9438dc8fc8, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:36.813204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0592b10-7c04-4e5c-9a97-b07cf0da7c6a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:36.852104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:36.852104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 191/400] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 10 11:03:36 2022] PUT /identity/v3/projects/d858077827a543fd96cbd471437ac078/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:36.852104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d667f-7dbe-4a94-ae9b-b2707df1b5a3 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.852104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d667f-7dbe-4a94-ae9b-b2707df1b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.852104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d667f-7dbe-4a94-ae9b-b2707df1b5a3 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.852104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 192/401] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:36.858745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.859078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.859294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.879854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:36.881251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:36.946558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 210/402] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 10 11:03:36 2022] PUT /identity/v3/projects/bcc0187a24b848949bcd600864be9608/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:03:36.948662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73282a3-0c9a-4047-ba6c-cc6bf2ac7847 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.949167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73282a3-0c9a-4047-ba6c-cc6bf2ac7847 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.949639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73282a3-0c9a-4047-ba6c-cc6bf2ac7847 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.951538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 211/403] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:36.957460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:36.957682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:36.958470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:36.984428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 193/404] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:36.996963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:36.998277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:37.007764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.084939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 212/405] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:37.111822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.152622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31aa4f-9de0-4316-95f0-a3f5f12616ee None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.187757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:37.198412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.199195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.199599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.199871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.213406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:37.219942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:37.220120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:37.220967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:37.221745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:37.225457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 194/406] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:37.228489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.243657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2cdc8a-2bdd-4856-b674-55626dc860b1 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.283390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.284557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.285081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.285541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.296760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:37.302016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:37.302873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:37.303666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:37.307965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 213/407] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:37.313848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.354279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be0fb8-0d94-409d-a452-a1d692438c2c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.400792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.401398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.401673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.401943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.402662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:37.402969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:37.404396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:37.419851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 195/408] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:37.421132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.478579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:37.516953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7028446b-46c7-4ee7-85cc-48c14b0f5bcb None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.539491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e903af-3e6e-4b5e-85a0-8dc883a300e0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.578476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.579103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.579382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.579644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.581095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:37.583104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:37.589486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 196/409] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:37.591907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.609238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.610490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.610963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.611426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.612345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:37.612845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:37.614408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:37.631193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 214/410] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:37.637112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.662177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:37.707514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9362c-5cb9-47f7-ae14-93fe3c704c2f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.727858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:37.746940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.747576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.747850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.748149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.749818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:37.751338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:37.760038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 197/411] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:37.760038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.852709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900be10-fd35-4313-9d06-8c2c093427f2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.891035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f93f6-a319-4eee-80c5-7ffadc436108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.891582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f93f6-a319-4eee-80c5-7ffadc436108 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.891862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f93f6-a319-4eee-80c5-7ffadc436108 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.892217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f93f6-a319-4eee-80c5-7ffadc436108 None admin] PATH_INFO: `/v3/groups/e7b9aff6911e40ca9b5ba8505f354745` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.893344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f93f6-a319-4eee-80c5-7ffadc436108 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b9aff6911e40ca9b5ba8505f354745)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:37.894776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f93f6-a319-4eee-80c5-7ffadc436108 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:37.896681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 215/412] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/groups/e7b9aff6911e40ca9b5ba8505f354745 => generated 229 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:37.897867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5d5f-eb54-48fa-8288-789be29d2d74 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.898249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5d5f-eb54-48fa-8288-789be29d2d74 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.898502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5d5f-eb54-48fa-8288-789be29d2d74 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.903839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 216/413] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:37.904630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c5d5f-eb54-48fa-8288-789be29d2d74 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:37.926989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2298c-1130-4682-a9db-9c8fefd41634 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:37.973659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63c226-ebda-4045-8851-9a2ec382ceb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:37.974970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c226-ebda-4045-8851-9a2ec382ceb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:37.975452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c226-ebda-4045-8851-9a2ec382ceb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:37.975922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63c226-ebda-4045-8851-9a2ec382ceb0 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:37.979643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c226-ebda-4045-8851-9a2ec382ceb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0187a24b848949bcd600864be9608, group_id=c4bcdeb5b770407895618e9438dc8fc8, role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:37.981681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63c226-ebda-4045-8851-9a2ec382ceb0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:37.993345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:38.017324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c5d5f-eb54-48fa-8288-789be29d2d74 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.050898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.052194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.052692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.053157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] PATH_INFO: `/v3/projects/1f5409bf396e4a7cbe05519266660a63` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.055104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] RBAC: Authorizing `identity:get_project(project_id=1f5409bf396e4a7cbe05519266660a63)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:38.055213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 198/414] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 10 11:03:37 2022] PUT /identity/v3/projects/bcc0187a24b848949bcd600864be9608/groups/c4bcdeb5b770407895618e9438dc8fc8/roles/22011eb257084dc58e55ad842a3a3292 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:38.056593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.056968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.057302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.057571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:38.059885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 217/415] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:37 2022] GET /identity/v3/projects/1f5409bf396e4a7cbe05519266660a63 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:38.066921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.077172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:38.078512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:38.133700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:38.170138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 199/416] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:38.194292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.202712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f5591-bfd2-4af4-a0bb-5ff979ca96c0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.249007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.249849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.250132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.250407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] PATH_INFO: `/v3/projects/1f5409bf396e4a7cbe05519266660a63/groups/e7b9aff6911e40ca9b5ba8505f354745/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.255038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] RBAC: Authorizing `identity:create_grant(project_id=1f5409bf396e4a7cbe05519266660a63, group_id=e7b9aff6911e40ca9b5ba8505f354745, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:38.256549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:38.309346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eeb97f-8e69-42e0-9ba9-f43b45b04663 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.333144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 218/417] 10.222.0.13 () {58 vars in 1442 bytes} [Wed Aug 10 11:03:38 2022] PUT /identity/v3/projects/1f5409bf396e4a7cbe05519266660a63/groups/e7b9aff6911e40ca9b5ba8505f354745/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:38.348666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.349808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.350344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.350863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.364889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:38.376106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:38.377059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:38.377835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 200/418] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:38.379381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.393180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:38.501536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a5411-5f85-4a73-851b-3c4730687e32 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.539440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.539922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.540237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.540512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.541180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:38.541483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:38.542886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:38.552172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 219/419] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:38.558673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:38.596185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:38.666295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db9609-05a4-434d-a12b-3647a5166bb6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:38.702767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:38.703434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:38.703717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:38.703988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:38.779915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:38.781366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:38.829589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:38.917636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 201/420] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:38.922937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.044444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.108440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b5f4-9d45-47c8-ad5e-66e8c5a9f327 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.154534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.155345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.155624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.155892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.160652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.162187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.164889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 220/421] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:38 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:39.170104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.306715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bdcef2-2694-4017-b892-8219ee5e76e4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.352716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a388e99-2844-4494-980d-27f437fb3c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.353346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388e99-2844-4494-980d-27f437fb3c8c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.353620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388e99-2844-4494-980d-27f437fb3c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.353887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a388e99-2844-4494-980d-27f437fb3c8c None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.356725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a388e99-2844-4494-980d-27f437fb3c8c None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.358192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a388e99-2844-4494-980d-27f437fb3c8c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.360127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 202/422] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:39.368060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.488481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9902f43-5712-46c0-bc05-36d7b5ca3cc3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.527113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.527778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.528085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.528360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.529115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=d858077827a543fd96cbd471437ac078) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:39.529423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:39.530856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e1ef3-a728-47a9-8fc7-866d3b07db67 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:39.535969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.541588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 221/423] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=22011eb257084dc58e55ad842a3a3292 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:39.618332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2d543-9e58-458f-b7a1-1172bae113da None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.618677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2d543-9e58-458f-b7a1-1172bae113da None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.618878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2d543-9e58-458f-b7a1-1172bae113da None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.620552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 203/424] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:39.623336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb588ca-98f2-428d-b22d-3f676002dbbe None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.623621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb588ca-98f2-428d-b22d-3f676002dbbe None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.623897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb588ca-98f2-428d-b22d-3f676002dbbe None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.625742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 222/425] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:39.626443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.626730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.627119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.631551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.631926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.632274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.635738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.648795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:39.650018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:39.654669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:39.655836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:39.728677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 204/426] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:39.771135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.792321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 223/427] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:39.822072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:39.918662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24157fd8-2306-4cde-af23-b6b9e01e6e80 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.957454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:39.958088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:39.958367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:39.958637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:39.959649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55599c70-4f4d-4656-8e96-83359995dc05 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:39.960736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.973566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:39.978582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:39.980657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:39.981773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 205/428] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:39.995063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:39.999317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.004607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.005319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.005657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.005987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.024372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:40.030236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:40.032278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:40.033488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 224/429] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:40.037518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.140635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b636e52-3c40-4466-ab3b-2d2653dc4c05 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.164441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb976d-b8b1-4f93-bb91-8b06714da499 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.190212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.190956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.191352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.191723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.193162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:40.193595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.195826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.206557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.207151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.207380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 206/430] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:40.208200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.208502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.209185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:40.209494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.211308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.213046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.224724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 225/431] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:40.230975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.285097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.308482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.338694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbd2ba-da33-4907-9da3-e5d5876af0ec None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.377771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68877ac5-e11f-46cc-a4ee-0ddd9ccde6b4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.391541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5d295-d38a-46a0-823f-bc1a13881c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.392937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d295-d38a-46a0-823f-bc1a13881c29 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.393762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d295-d38a-46a0-823f-bc1a13881c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.394237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5d295-d38a-46a0-823f-bc1a13881c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.421276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29aeeb-750b-4154-a73c-3fdc5ff795d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.421883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29aeeb-750b-4154-a73c-3fdc5ff795d0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.422161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29aeeb-750b-4154-a73c-3fdc5ff795d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.422433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29aeeb-750b-4154-a73c-3fdc5ff795d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.488783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5d295-d38a-46a0-823f-bc1a13881c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.490877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5d295-d38a-46a0-823f-bc1a13881c29 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.512762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29aeeb-750b-4154-a73c-3fdc5ff795d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.514858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29aeeb-750b-4154-a73c-3fdc5ff795d0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.565950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.582494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.612093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 207/432] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:03:40.613348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3abfac-d117-4688-9468-ff5147b33219 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.614048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3abfac-d117-4688-9468-ff5147b33219 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.614666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3abfac-d117-4688-9468-ff5147b33219 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.616722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 208/433] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:40.617987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3abfac-d117-4688-9468-ff5147b33219 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.645865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 226/434] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:40.647302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.647589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.648434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.666549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:40.667711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:40.730505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3abfac-d117-4688-9468-ff5147b33219 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.736285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 227/435] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:40.741193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.769647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1243d-c84d-4efb-a3b0-9cbd154bf2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.770275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1243d-c84d-4efb-a3b0-9cbd154bf2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.770546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1243d-c84d-4efb-a3b0-9cbd154bf2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.770814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1243d-c84d-4efb-a3b0-9cbd154bf2c0 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.771904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1243d-c84d-4efb-a3b0-9cbd154bf2c0 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.773889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1243d-c84d-4efb-a3b0-9cbd154bf2c0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.775482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 209/436] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:40.777304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf93a-88b9-46db-b76d-8f5e1cae212a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.777734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf93a-88b9-46db-b76d-8f5e1cae212a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.778009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf93a-88b9-46db-b76d-8f5e1cae212a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.779335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 210/437] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:40.779898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcf93a-88b9-46db-b76d-8f5e1cae212a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.805836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.830766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:40.849586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f8e8e-f908-45ae-ac8b-b040a150cee9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.880596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcf93a-88b9-46db-b76d-8f5e1cae212a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:40.884091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.884713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.884991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.885262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] PATH_INFO: `/v3/projects/d858077827a543fd96cbd471437ac078` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.886383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] RBAC: Authorizing `identity:get_project(project_id=d858077827a543fd96cbd471437ac078)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:40.887831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:40.889490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 228/438] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/projects/d858077827a543fd96cbd471437ac078 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:40.890244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:40.915505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb4aff0-ffff-4126-ab02-43ec169ecda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:40.916153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4aff0-ffff-4126-ab02-43ec169ecda9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.916434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4aff0-ffff-4126-ab02-43ec169ecda9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.916701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb4aff0-ffff-4126-ab02-43ec169ecda9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.926392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb4aff0-ffff-4126-ab02-43ec169ecda9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-8cb4aff0-ffff-4126-ab02-43ec169ecda9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:40.931766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:40.932907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:40.933703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 211/439] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:40.934367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:40.934367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:40.934481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:40.952663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:40.953864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:40.991899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40f98-16d1-43d2-85a1-b1acbc39186a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.020298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 212/440] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:41.021130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.026158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.026781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.027054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.027320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.028443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.029868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.031420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 229/441] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:40 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:41.032296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.050231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.050788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.129819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1e184-7890-4d45-9dd3-a8a4ce5baff0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.131829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65612f4-23ae-4969-9c67-dc17ebed8973 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.163698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.164324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.164599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.164865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.165517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:41.165819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.166401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.167002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.167225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.167316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.167530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.168651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.170100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.171684 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 213/442] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:41.172541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.180660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 230/443] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:41.181943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.254008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.260916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.271699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8a371-0b02-4a72-8a90-cb5118951f6a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.281389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91f669-3bb3-407e-9bc6-19f579262c04 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.305373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.305962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.306240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.306508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.307218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=bcc0187a24b848949bcd600864be9608) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:41.307520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.308965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.313858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.314484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.314620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 214/444] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:41.314828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.315028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.315373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.327748 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:41.333294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:41.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:41.335079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 231/445] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:41.335831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.438823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd08dc-71d9-4ae2-93c2-f77cd33a99de None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.455509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414c58f-8a52-4f15-be74-80e9ac76c6b8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.483089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.483823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.484168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.484451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.485177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=33b4484deb1044fdb1ab7ae8ebc74f8f, scope.project.id=d858077827a543fd96cbd471437ac078) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:41.485515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.487055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.494063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 215/446] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=d858077827a543fd96cbd471437ac078&role.id=33b4484deb1044fdb1ab7ae8ebc74f8f => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:41.495100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.495953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.496600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.496880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.497146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.508447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.514027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.587464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.588981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.627037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeabf61-68bd-4dfb-9610-51c59fa28c27 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.667804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.668443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.668722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.670795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.670795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:41.670795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.671407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.681281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 216/447] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:41.686645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.707398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 232/448] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:41.713365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.756806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.780179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:41.817239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16304d8c-46d1-4cd1-96b1-efcdc2e74695 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.831124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c31f8bb-7d7f-4282-a1e4-d31b218884c4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:41.864798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.864798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.864798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.864798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.871283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:41.871917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:41.874441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:41.874441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] PATH_INFO: `/v3/projects/bcc0187a24b848949bcd600864be9608` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:41.874441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] RBAC: Authorizing `identity:get_project(project_id=bcc0187a24b848949bcd600864be9608)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.875423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:41.877258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 233/449] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/projects/bcc0187a24b848949bcd600864be9608 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:41.882959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:41.948791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:41.950239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:42.016197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35389f-5032-44f6-9c46-e0c65ce6c242 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:42.028346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:42.030826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:42.057010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 217/450] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:03:42.060791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:42.061431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:42.061912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:42.062028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] PATH_INFO: `/v3/groups/c4bcdeb5b770407895618e9438dc8fc8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:42.062828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:42.063090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] RBAC: Authorizing `identity:get_group(group_id=c4bcdeb5b770407895618e9438dc8fc8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:42.064566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:42.066120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 234/451] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:41 2022] GET /identity/v3/groups/c4bcdeb5b770407895618e9438dc8fc8 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:42.072198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:42.179273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912e3dd-43cd-4195-9017-738883242e79 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:42.192090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e304034-578b-4de7-9198-ba61b0a00089 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:42.214211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:42.214846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:42.215148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:42.215415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] PATH_INFO: `/v3/projects/1f5409bf396e4a7cbe05519266660a63` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:42.216580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] RBAC: Authorizing `identity:get_project(project_id=1f5409bf396e4a7cbe05519266660a63)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:42.218007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:42.219929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 218/452] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:03:42 2022] GET /identity/v3/projects/1f5409bf396e4a7cbe05519266660a63 => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:42.224914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:42.225136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:42.225743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:42.226675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:42.226952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:42.227864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] RBAC: Adding query filter params (group.id=c4bcdeb5b770407895618e9438dc8fc8, role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=bcc0187a24b848949bcd600864be9608) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:42.228210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:42.229627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:42.235394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 235/453] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:42 2022] GET /identity/v3/role_assignments?group.id=c4bcdeb5b770407895618e9438dc8fc8&scope.project.id=bcc0187a24b848949bcd600864be9608&role.id=22011eb257084dc58e55ad842a3a3292 => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:42.268310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:42.330140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8304fa-dbe3-4a12-b21d-dbe5bfd71700 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:42.366852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e3f9d-2045-4c48-a2d5-fcd01356d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:42.367689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e3f9d-2045-4c48-a2d5-fcd01356d620 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:42.367965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e3f9d-2045-4c48-a2d5-fcd01356d620 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:42.368264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e3f9d-2045-4c48-a2d5-fcd01356d620 None admin] PATH_INFO: `/v3/groups/e7b9aff6911e40ca9b5ba8505f354745` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:42.369450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e3f9d-2045-4c48-a2d5-fcd01356d620 None admin] RBAC: Authorizing `identity:get_group(group_id=e7b9aff6911e40ca9b5ba8505f354745)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:42.370889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184e3f9d-2045-4c48-a2d5-fcd01356d620 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:42.372571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 219/454] 10.222.0.13 () {56 vars in 1180 bytes} [Wed Aug 10 11:03:42 2022] GET /identity/v3/groups/e7b9aff6911e40ca9b5ba8505f354745 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:42.378330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:42.422083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:42.481862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf2947-08a5-4513-968f-d16a2d7e4542 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:42.515724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:42.516385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:42.516660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:42.516930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:42.517651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] RBAC: Adding query filter params (group.id=e7b9aff6911e40ca9b5ba8505f354745, role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=1f5409bf396e4a7cbe05519266660a63) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:42.517949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:42.519361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9dfa0-0e67-4702-9d95-9058627fd8d5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:42.525924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 236/455] 10.222.0.13 () {56 vars in 1376 bytes} [Wed Aug 10 11:03:42 2022] GET /identity/v3/role_assignments?group.id=e7b9aff6911e40ca9b5ba8505f354745&scope.project.id=1f5409bf396e4a7cbe05519266660a63&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:44.901826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9c922-97e2-4b21-bd8a-c53cd7a7a89e None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.902348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9c922-97e2-4b21-bd8a-c53cd7a7a89e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.902422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9c922-97e2-4b21-bd8a-c53cd7a7a89e None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.903973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 220/456] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:44.908971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd8b8-9743-416e-aaeb-65d8dc4adb49 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:44.909374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd8b8-9743-416e-aaeb-65d8dc4adb49 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:44.909538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fd8b8-9743-416e-aaeb-65d8dc4adb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:44.932597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a44fd8b8-9743-416e-aaeb-65d8dc4adb49 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:44.933842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a44fd8b8-9743-416e-aaeb-65d8dc4adb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:45.030180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 237/457] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:45.066480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9c922-97e2-4b21-bd8a-c53cd7a7a89e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:45.146913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9889e8-c9ef-437a-804a-63f1d106bd0d None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.147234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9889e8-c9ef-437a-804a-63f1d106bd0d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.147465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9889e8-c9ef-437a-804a-63f1d106bd0d None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.150172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 238/458] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:45.159571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.159571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.159571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.193349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:45.194543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:45.205499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9c922-97e2-4b21-bd8a-c53cd7a7a89e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.222166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.249146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.264501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.265444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.265927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.266399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.278176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.279878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:45.297685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:45.298634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:45.299461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:45.299461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:45.299461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:45.301739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 221/459] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:45.313324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:45.354393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 239/460] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:45.385913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:45.447235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c962a53-e27d-4858-afdd-2845cd98e3a6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.488827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.489439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.489715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.489986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.490636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:45.490938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.492878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.512194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 222/461] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:45.518575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:45.562080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.591661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677b47a-556d-4119-8124-afdc1cf48016 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.629880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.630836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.631317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.631760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.642565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.644278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.647016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e9076-96d6-4c35-82f3-61f7001aca29 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-bd51d95e-cfcf-4bc3-8d56-2c32f8a56348 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:45.659937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:45.660897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:45.661800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:45.661800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:45.661800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:45.664976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 240/462] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:45.666358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537b68-9e19-4248-9b2d-8ad027b5134c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.666824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537b68-9e19-4248-9b2d-8ad027b5134c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.667259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537b68-9e19-4248-9b2d-8ad027b5134c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.668932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 241/463] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:45.673823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.674382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.674836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.690364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a981486-1952-4214-b5d4-4c3470439c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.691050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a981486-1952-4214-b5d4-4c3470439c97 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.691254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a981486-1952-4214-b5d4-4c3470439c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.691609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a981486-1952-4214-b5d4-4c3470439c97 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.692201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a981486-1952-4214-b5d4-4c3470439c97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.693810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a981486-1952-4214-b5d4-4c3470439c97 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.696896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:45.698140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:45.707853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.764245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 242/464] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:45.765161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:45.791571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 223/465] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:45 2022] POST /identity/v3/users => generated 246 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:45.792980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100fe1-c697-4200-977f-cbb514cb6b94 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.793266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100fe1-c697-4200-977f-cbb514cb6b94 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.793540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4100fe1-c697-4200-977f-cbb514cb6b94 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.794875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 224/466] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:45.795506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100fe1-c697-4200-977f-cbb514cb6b94 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:45.890129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.900684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef81cd5-5b7f-4de4-9d1d-870e22a8925e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.919888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4100fe1-c697-4200-977f-cbb514cb6b94 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:45.950378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.950915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.951084 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.951505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.952328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:45.952786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.954150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9aff3-41a9-42d8-8805-8cee6d331aec None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.971024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 243/467] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:45.971463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:45.972474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.972611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:45.972992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.973250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:45.973349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.973768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:45.981940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:45.987148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:45.988655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:45.994012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:45.995177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:46.004709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:46.005620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:46.006518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:46.006518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:46.006518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:46.008313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 225/468] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:46.009238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.088665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 244/469] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:46.089990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.136945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a654079d-a321-4c0a-b299-d83073eb3707 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.172442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:46.178555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.179951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.180255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.180668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.181894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.183417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.207660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb510d9-5139-42e3-bae7-c61f00ca0e9d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.245448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.246064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.246342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.246613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.247251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:46.247551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.248942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.250584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:46.261056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 245/470] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:46.261953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.307169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 226/471] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:46 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:46.308055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.434404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b796e4-d82b-4dd8-85d4-c16a24d41ad7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.436306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236b6a-6b39-4589-a61f-e59286de5b69 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.480506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af1cd9-ae20-41ad-b106-2835a1d4329a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.481037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af1cd9-ae20-41ad-b106-2835a1d4329a None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.484027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af1cd9-ae20-41ad-b106-2835a1d4329a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.484027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af1cd9-ae20-41ad-b106-2835a1d4329a None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.484027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af1cd9-ae20-41ad-b106-2835a1d4329a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.484479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af1cd9-ae20-41ad-b106-2835a1d4329a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.510634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.511288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.515954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.515954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.525224 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.526676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:46.543041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:46.548859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:46.550369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:46.550369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:46.550369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:46.550369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 246/472] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:46.550796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.602837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:46.645499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 227/473] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:46 2022] POST /identity/v3/users => generated 248 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:46.646883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52602466-0f6d-4705-93ae-700cf2bc773b None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.647579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52602466-0f6d-4705-93ae-700cf2bc773b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.647579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52602466-0f6d-4705-93ae-700cf2bc773b None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.659447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 228/474] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:46.659447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.659447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.659447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.664226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd326eff-f8cb-4f1e-92f8-ad7740a511d7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.679112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:46.681918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:46.703110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.703885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.704116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.704395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.705090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:46.705395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.706917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.721257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 247/475] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:46.725325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:46.727294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.764634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 229/476] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:46.788370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:46.837491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96442071-2af6-4db9-9459-cacc4c97221c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.846244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:46.875232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.875849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.876167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.876441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.877093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.878652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.908956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40891c04-9993-4344-9289-6a7b7dcb5bd4 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:46.950989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:46.951600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:46.951851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:46.952165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:46.953129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:46.954526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:46.966887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 248/477] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:46 2022] POST /identity/v3/users => generated 249 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:47.011575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.sql.core [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 11:03:47.011575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 11:03:47.011575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [parameters: {'id': '8dedb07922164d4d943ca62d3e605160', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 11:03:47.011575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80628) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application result.read() Aug 10 11:03:47.031431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return ref.to_dict() Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:03:47.032376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application yield resource Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise self.value Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application session.commit() Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 11:03:47.033288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self.session.flush() Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application compat.raise_( Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 11:03:47.034859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:03:47.035691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application result.read() Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:03:47.036550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [parameters: {'id': '8dedb07922164d4d943ca62d3e605160', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:47.037336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 10 11:03:47.040508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:47.041836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 230/478] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 10 11:03:46 2022] POST /identity/v3/roles => generated 144 bytes in 245 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:47.041914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:47.159209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c770351-da38-4c1c-8b9a-52fc5cc613fa None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:47.196305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e53f1b-81ca-4bba-949c-69df9cbac112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:47.196935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e53f1b-81ca-4bba-949c-69df9cbac112 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.197211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e53f1b-81ca-4bba-949c-69df9cbac112 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.197479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e53f1b-81ca-4bba-949c-69df9cbac112 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.207951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e53f1b-81ca-4bba-949c-69df9cbac112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-36e53f1b-81ca-4bba-949c-69df9cbac112 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:47.213764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:47.214636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:47.215435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 249/479] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:47.220762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:47.262103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:47.333843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b58c83-663b-401b-991a-871f16a9a1c6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:47.375667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:47.376317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:47.376723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:47.376990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:47.377655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:47.377952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:47.379369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e9849-0d10-42d5-8989-2e3bfc1ae95f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:47.388647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 231/480] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:48.386084 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e313b-82f0-4689-989e-32a5335c279c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.386577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e313b-82f0-4689-989e-32a5335c279c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.386652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e313b-82f0-4689-989e-32a5335c279c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.388218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 250/481] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:48.394710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.395105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.395270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.424929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:48.426121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:48.513798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 232/482] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:48.536428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e313b-82f0-4689-989e-32a5335c279c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.578611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:48.690908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e313b-82f0-4689-989e-32a5335c279c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:48.736496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf36c1-2085-4f39-96b7-e1e7852b9b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:48.737134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf36c1-2085-4f39-96b7-e1e7852b9b59 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.737413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf36c1-2085-4f39-96b7-e1e7852b9b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.737683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf36c1-2085-4f39-96b7-e1e7852b9b59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.752135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf36c1-2085-4f39-96b7-e1e7852b9b59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-47cf36c1-2085-4f39-96b7-e1e7852b9b59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:48.756991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:48.758157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:48.758945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:48.763017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 251/483] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:48.764429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.767957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967df09-da85-40bd-9363-9fcb632f3d75 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.768267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967df09-da85-40bd-9363-9fcb632f3d75 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.768544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967df09-da85-40bd-9363-9fcb632f3d75 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.770134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 252/484] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:48.776024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.776298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.776571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.800393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:48.813446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:48.814644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:48.911418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561ad05-7c38-4d57-b200-70029bb8b8b2 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:48.950205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:48.950722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:48.951195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:48.951468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:48.952156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:48.952504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:48.953867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:48.964099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 233/485] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:48.977815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:48.992152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 253/486] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:49.051269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.080448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.144359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.167670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c19ed1-48d2-44ee-a884-73d4e433ea5a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.183162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd42c24-36d4-48ab-a5f9-1e928716ddf9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.205409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.206324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.206562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.206839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.219089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.219809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.219932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.220267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.220605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.221505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.231881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:49.236643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:49.237677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:49.238662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-f7a77182-44b2-4a09-9385-578134b97627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:49.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:49.240279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:49.241124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:49.241124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:49.241124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:49.241124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 234/487] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:49.241124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee571c4a-6bcf-43e5-a0e0-96b0f48c043a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.241124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee571c4a-6bcf-43e5-a0e0-96b0f48c043a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.241546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee571c4a-6bcf-43e5-a0e0-96b0f48c043a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.245313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 254/488] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:49.247471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.248351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 235/489] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:49.253247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee571c4a-6bcf-43e5-a0e0-96b0f48c043a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.322198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.353012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f9261-4a03-44a7-960a-06c74c6ab899 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.398581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee571c4a-6bcf-43e5-a0e0-96b0f48c043a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.407607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.408389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.408767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.409134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.410340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:49.410747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.412936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884de42e-a03b-4ded-94ad-ad63220dcc29 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.435711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 255/490] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:49.435711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.435711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.435711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.442895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.443695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.444157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.444632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.445519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:49.445962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.447542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.455123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:49.456562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:49.456982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 236/491] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:49.458077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.465472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.548031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 256/492] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:49.548031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.607487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e4031-55e6-4d6e-821f-e4a9a8879f71 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.622192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.643232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.644229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.644736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.645202 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.654537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.656387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:49.669959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:49.670954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:03:49.671922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:49.672449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab7670-104f-41d0-b7f4-d6d2994d7800 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.678850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 237/493] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:49.680936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.714087 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.714866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.715163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.715440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.729675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.731434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.749144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:49.753259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:49.754175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:49.755414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:49.755414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:49.755414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:49.759801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 257/494] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:49.760707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.796832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2a3433-f393-448c-b627-bb2f24a3725b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.838115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.838852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.839406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.839406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.849479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:49.855108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:49.856027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:49.856845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 238/495] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:49.858506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.884907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e441ae-b3ab-4980-a933-442cd68e20c4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.900205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:49.920773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:49.921382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.921659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.921932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.922719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:49.923029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:49.924487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c95d20-6f61-4544-a88c-434bbf13efb0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:49.961937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 258/496] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:49.963252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ae65c-9429-4461-97fb-23d5cb521168 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:49.963536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ae65c-9429-4461-97fb-23d5cb521168 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:49.963803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ae65c-9429-4461-97fb-23d5cb521168 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:49.965227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 259/497] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:49.965759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ae65c-9429-4461-97fb-23d5cb521168 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:49.975561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778f9738-d38b-4263-a77d-0b4faf808c87 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:49.999923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.010258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.015868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.015868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.015868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.015868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:50.015868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.015868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.025405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 239/498] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:50.026306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.072385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ae65c-9429-4461-97fb-23d5cb521168 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.105339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.105931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.106207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.107132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:50.107435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.108836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51ab92-0301-439e-b2e8-f9bfefab9411 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.116400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 260/499] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:50.118127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b5214-3c51-469c-a5f4-c4332d130599 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.118406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b5214-3c51-469c-a5f4-c4332d130599 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.118749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b5214-3c51-469c-a5f4-c4332d130599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.124837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba616d-b2bd-4037-92a8-efa7bbd64dcd None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.136346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5e4b5214-3c51-469c-a5f4-c4332d130599 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:50.137538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5e4b5214-3c51-469c-a5f4-c4332d130599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:50.158184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.158780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.159053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.159324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.162827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.203134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 261/500] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:50.204238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9416-334a-42c2-ae45-9b6ab68c6e0a None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.204792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9416-334a-42c2-ae45-9b6ab68c6e0a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.205523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9416-334a-42c2-ae45-9b6ab68c6e0a None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.207755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 262/501] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:50.209550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.212571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9416-334a-42c2-ae45-9b6ab68c6e0a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.228933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.230345 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.313371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9416-334a-42c2-ae45-9b6ab68c6e0a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.328472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 240/502] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:50.329312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.348320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.348944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.349220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.349494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.357150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.358602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-800161d4-034d-453b-baba-8c71fa74111c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:50.371439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:50.372486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:50.373390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 263/503] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:50.374082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.374082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.374252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.391899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:50.393117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:50.410616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.420870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.438147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733cdf3b-469c-48af-8498-bb9b63257af3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.461268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 264/504] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:50.462096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.476494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.477623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.477885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.478162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.488816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.490240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:50.504848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:50.505816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:50.506656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:50.506656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:50.506656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:50.506656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 241/505] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:50.506865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.564117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d07e3-4ee5-4762-a478-d6f4faf6742f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.597605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.598315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.598510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.598784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.607600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57952c-3f43-41ed-b681-a6c106dbfbac None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.609355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:50.614046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:50.614912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:50.615700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 265/506] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:50.616575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.631307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.644612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.645234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.645509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.649471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.649471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.657104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.658546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:50.671931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:50.678992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:50.680349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:50.680349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:50.680349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:50.680349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 242/507] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:50.680349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.715896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b410fc62-d788-426b-9893-ec3dbe0a98ef None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.748984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.749505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.749777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.750045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.750780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:50.751072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.752512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.769355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 266/508] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:50.770225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.774496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c4e7b-ee48-4b98-ad4a-6da902f20600 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.808184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.808810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.809087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.809356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.820383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.821841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:50.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:50.838317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:50.844190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:50.844190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:50.844190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:50.844190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 243/509] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:50.844190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.844482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.873528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e614c4b-11b4-4453-8ec0-4143a02f74d1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.878988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:50.908821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.909418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.909688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.909953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.910577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:50.910872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.912265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.924962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 267/510] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:50.925760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:50.937956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a015e-3584-4da7-b3e3-806d46134c36 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:50.971259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:50.971848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:50.972158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:50.972428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:50.973077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:50.973373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:50.974730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:50.983329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 244/511] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:50.984127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.026557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1ab45-e107-44cb-8600-98a72a68b645 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.054672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.059600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.060241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.060515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.060781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.061397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:51.061690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.063027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.074047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 268/512] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:51.075093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.082046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.084788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af95bb-b538-4fab-a916-5f6915d2a3db None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.121602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.122204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.122477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.122746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.179162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d38ec-b02a-4de9-83ce-15d4ced54f93 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.190976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.192497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.212897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.213494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.213768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.214032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.214655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:51.214952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.216440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.228847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 269/513] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:51.229679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.282553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.286120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.290129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 245/514] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:51.290941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.328990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265bdd91-9a4f-4f4c-afad-b5764ca70d9d None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.368886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.369660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.369938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.370205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.377704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.379137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.386809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834c777a-7fa0-42b7-b75a-95cf3cb03fd6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:51.391565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:51.392477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:03:51.393360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:51.394053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 270/515] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:51.395070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.422236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.423035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.423148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.423386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.433374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.434814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:51.449487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:51.450536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:51.451368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:51.451368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:51.451368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:51.451368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 246/516] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:51.451368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.493638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40bb78-32cb-4b1f-92e2-ba57e39a89d8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.504804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.506105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.526360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.526998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.527261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.527534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.534073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.535458 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:51.547655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:51.548640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 271/517] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:51.549533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.550050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee856a-5bb6-4797-be59-e23f699c9347 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.583901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.584526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.584804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.585074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.585791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.587237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.650828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b816ba-b5d4-45df-bda5-94c3575bc211 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.683710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.684378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.684661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.685015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.695641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.697199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.717675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 247/518] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:03:51 2022] POST /identity/v3/users => generated 247 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:03:51.717675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:51.717915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:51.718907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:51.719950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:51.719950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:51.719950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:51.719950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 272/519] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:51.719950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.746939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.823516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc419-5593-4684-9a40-91724ed1d8c9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.830766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24373ff7-d2bf-4c4b-aaea-337f6d9387b3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:51.833170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:51.861924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.862919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.863240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.863514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.864288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:51.864593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.866005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.870974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:51.872031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:51.872031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:51.872210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:51.872755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:51.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:51.875192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:51.889089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 273/520] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:51.889929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.912024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 248/521] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:51.912024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:51.990057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.006557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35ee24-2204-4b58-9c7e-8baa410e1ea8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.013555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94077f9b-4454-4a7d-90f9-1eeea79cb492 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.040472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.041092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.041342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.041617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.042306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.042607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.044049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.057383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.057995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.058275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.058546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.059187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.059487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.060896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.068435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 274/522] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:52.068435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.073090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 249/523] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:52.074135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.105147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.174064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df728059-760f-46b7-a3c2-550b5dd3a485 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.182841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3c767-0608-4c2d-89aa-d5fd86088044 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.210815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.211603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.212031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.212306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.219075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.219604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.219707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.219952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.220286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.221062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:52.233274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:52.234265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 275/524] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:52.235147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.239557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.297274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.298731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.329248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.349436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd5522-7ceb-4864-aac6-39dbb4bb58d2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.389163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.389772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.390049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.390323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.391044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=462638e8386c49668290954c0211728c) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.391352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.392828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.400057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 276/525] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/role_assignments?user.id=462638e8386c49668290954c0211728c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:52.404998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.421858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 250/526] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:52.422800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.503885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.543293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d053861-e642-4f69-a6dd-8e838216ccc7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.544492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d380b4b-a224-48c7-b269-bbc893f45a00 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.574490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.583400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.583943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.584353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.584587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.585369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.585676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.586035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.586877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.587215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.587557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.588032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.595429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.597130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.610621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 251/527] 10.222.0.13 () {56 vars in 1139 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:52.610621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:52.614911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:52.615743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:52.617170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:52.619544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 277/528] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:52.620734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.715041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2caba5-cfec-4687-9498-3d2cb607bf6b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.725416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc28f8-5674-4cba-9a32-f8c1912220d7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.751041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.751454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.751731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.752026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.756480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.762679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.764177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.764949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.766022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.766492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.767317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:52.780613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:52.781550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:52.782480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:52.782480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:52.782480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:52.786452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 252/529] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:52.787303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.824565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:52.855006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.856402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.917731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec663a1c-8bf5-452e-8a9d-c8ede86988a8 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:52.965834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:52.966803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:52.967308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:52.967767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:52.967954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 278/530] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:03:52.968852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:52.968980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:52.969462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:52.971359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:52.996551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 253/531] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:52.998095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.002052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.089499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706a69-d21e-4ad4-a46a-10599b43045e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.107846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.114053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a9335-53d3-4403-9073-251e4426a4ad None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.136705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.137482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.137803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.138078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.139770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.140111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.141509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.157419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 279/532] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:53.158260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.160436 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.161537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.163786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.164126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.178204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.182155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:53.203032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:53.203989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:53.204862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:53.204862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:53.204862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:53.207921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 254/533] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:53.209358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.264206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.279568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3be2b-23a2-481a-824b-25a549a1d7c7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.324252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.325392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.325852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.326424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.327644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=3a95ab030c9940c9946c64c82567773c) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.328429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.330510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.332997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a9193-b067-4c07-8c08-2270aa3f6fc2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.344026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 280/534] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/role_assignments?user.id=3a95ab030c9940c9946c64c82567773c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:53.344026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.402113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.404511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.405811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.406318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.421972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.423798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:53.448591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:53.449523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:53.450360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:53.458690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 255/535] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:53.462489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.485681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.565948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db3865-39c2-41ad-b272-52ccde633d61 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.615709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.617255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.618089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.618562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.619426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.619923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.622135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.635517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 281/536] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:53.640816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.653135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe51a3de-dfff-47db-8d4b-8c4c02406f5c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.669175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.719937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.720948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.721423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.721919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.722804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.723302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.725090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.738953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 256/537] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:53.745177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.814592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:53.847953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc822978-c32b-445b-b224-e4f99663b269 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.851229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87f7d9-7964-4948-b8b7-88fa855995f0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:53.895524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.896096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:53.896239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.896426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.896685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.896892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.897343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.897814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.898778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=3cada6c06ef8427395eda888397efac4) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:53.899380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.901177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cc2e-e3c0-46c0-82b4-2abebc875eb2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.906864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:53.908319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:53.908563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 257/538] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/role_assignments?user.id=3cada6c06ef8427395eda888397efac4&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:53.912264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38556-f94d-41da-97fc-336f4d33dd01 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.912553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38556-f94d-41da-97fc-336f4d33dd01 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.912822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38556-f94d-41da-97fc-336f4d33dd01 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.915405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 258/539] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:53.922062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:53.922368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:53.922604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:53.936416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:53.937352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:53.938198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:53.941642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 282/540] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:53.943636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:53.969166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:53.970415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:54.089744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.103606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 259/541] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:54.124915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73efe996-39ef-459a-985d-b8a030001bd3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.128407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.172379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.172379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.172379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.172379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.172779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:54.172779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.172779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.193014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 283/542] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:54.199095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.201320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.248620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbce67b-2363-42cf-8e1c-7266ae101835 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.291151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.292132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.292828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.293290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.307595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:54.314331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:54.315251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:54.316061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:54.319105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 260/543] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:54.326312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.397086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d0818-2f48-4092-8243-49a367d2b707 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.430754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.440996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.447027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.447027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.447027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.447027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=52773c21d6e9406086113a9fd74d5f62) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:54.447027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.447027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.461516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 284/544] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/role_assignments?user.id=52773c21d6e9406086113a9fd74d5f62&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:54.495244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a3d92-8a52-4165-86bf-468dfc0063c6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.524464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.545999 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.546633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.546911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.547176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.547861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:54.548188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.549790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557df6cf-4ed9-48f7-912f-2c77b14ca8ad None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.568103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 261/545] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:54.574916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.647366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b5d5-eba3-4965-ba6a-92c3802d46f6 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.647731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b5d5-eba3-4965-ba6a-92c3802d46f6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.648034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b5d5-eba3-4965-ba6a-92c3802d46f6 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.649671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 262/546] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:03:54.655378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.655956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.656465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.676072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:54.677514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:54.709925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba53d5-c01f-4f5c-8bad-91c550cf6e5a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.753632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.754650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.755073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.755640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.761642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 263/547] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 10 11:03:54.767542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.769231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:54.784283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:54.785495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:54.786524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:54.786524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:54.786524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:54.788821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 285/548] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:54.790304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.794935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.853841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.912482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bb57a-de37-4893-ba62-d0330feadab7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.912906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f749bc0-072c-41c4-b031-acb0c3289ac2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:54.932090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:54.949951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.950886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.951366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.951832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.952701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:54.953231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:54.954890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:54.965536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:54.966114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:54.966384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:54.966658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:54.973456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 286/549] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:54.979757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:54.993412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:54.999841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:55.000796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:55.001607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:55.006408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 264/550] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:55.016993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.172853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a3098-a9c2-4029-b852-303c7dadc050 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.188695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:55.205950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b999684-686c-40e3-a1e2-6d6df29c625e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.228451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.229093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.229413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.229715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.245778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.245778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:55.268507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:55.269674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:55.270561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 287/551] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:03:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:03:55.277725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.278664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.279274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.279548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.279816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.280521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:55.280821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.282184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.291663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 265/552] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:55.301134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.364992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:55.456355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06761fb0-01de-40b7-a270-7b595e76239a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.478712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc539c-283b-4f04-8506-6083d90a3c16 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.512180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.513117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.513635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.514087 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.514947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:55.515527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.517268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.542277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.543517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.545029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.545507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.549426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 288/553] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:55.555357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.568133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.569908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:55.598093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:55.599249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:55.600313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:55.600313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:55.600313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:55.602825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 266/554] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:55.607656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:55.609327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.715526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:55.729410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105fbe9a-c13f-4141-a52c-f7483a3da57f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.733813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0702638-852a-4ed7-81af-796978977a96 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.768774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.769379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.769658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.769932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.770574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:55.770876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.772293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a1241-965b-4cd2-9e24-4608717d7c08 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.774282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27f0948-4ff7-45a4-9a70-68699d7e274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.774890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f0948-4ff7-45a4-9a70-68699d7e274a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.775170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f0948-4ff7-45a4-9a70-68699d7e274a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.775446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27f0948-4ff7-45a4-9a70-68699d7e274a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.789888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 267/555] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:03:55.791474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1b9fb-5cde-41d6-b815-c3b5743e1687 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.791952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1b9fb-5cde-41d6-b815-c3b5743e1687 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.792477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1b9fb-5cde-41d6-b815-c3b5743e1687 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.794505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 268/556] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:55.795477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1b9fb-5cde-41d6-b815-c3b5743e1687 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.848640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f0948-4ff7-45a4-9a70-68699d7e274a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.850086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27f0948-4ff7-45a4-9a70-68699d7e274a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.878000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:55.903838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1b9fb-5cde-41d6-b815-c3b5743e1687 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:55.942302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:55.943342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.943862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.944960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.953247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:55.955111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:55.958630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 289/557] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 10 11:03:55.959974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:55.960924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:55.961201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:55.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:55.970953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:03:55.971875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:55.974251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 269/558] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:55.975332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:55.985053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:55.986206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:55.988373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.061820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 290/559] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:56.062412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.088507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755d770-7d2e-48e0-ad8a-03624c20104d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.120335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.126975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.127906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.128430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.128857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.140149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.142119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:56.163805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.165463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:56.166374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:56.166374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.166374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:56.169330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 270/560] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.170427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.175873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e82b3f-0d4f-456c-9dfc-6e481b0dc1f4 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.215791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.216452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.216751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.217025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.217836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:56.218042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.219641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.242773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 291/561] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:56.243606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.254618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.311780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596e84d-d577-4a0d-b4cc-3f980b2762f0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.349171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.349812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.350094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.350367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.363786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:56.366935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5d882-f46c-43c4-88e9-01c0582d3d6b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:56.371254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.372292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:56.373501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 271/562] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.374913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.389780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.415844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.416498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.416777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.417046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.417708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:56.418763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.421494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.433971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 292/563] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:56.435093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.498516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f76a03-33b4-494d-82a2-d5be7cfe342c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.521079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.536675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.537301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.539707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.539707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.541381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9021fa8-0ec5-41a4-9e6e-5ccd6e612ec5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.549229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.550693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-bf5737da-7309-4452-a1ef-116d84c58b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:56.567255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.573894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 272/564] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75270014-1fe9-4c46-9eda-15612c6854e0 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75270014-1fe9-4c46-9eda-15612c6854e0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75270014-1fe9-4c46-9eda-15612c6854e0 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 273/565] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:56.574821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75270014-1fe9-4c46-9eda-15612c6854e0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.587368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.588087 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.588441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.588787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.589524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:56.589899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.591334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.599808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 293/566] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:56.605073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.641395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.694839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75270014-1fe9-4c46-9eda-15612c6854e0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.714097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a611dd-cc8a-421c-bee9-ef8663fca974 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.730274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.730913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.731184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.731452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.738066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.739501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.749697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.750247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.750525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.750811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-fc899459-16e3-4a5b-a202-ba6ec837a8e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:56.751037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.751891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:56.758010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:56.758572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:56.758572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.758721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 274/567] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.758800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.758887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.758887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.758887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.774310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 294/568] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:56.774310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.774600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:56.774600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:56.800904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.844579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 275/569] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:56.845449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.887457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:56.896504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf67e8-cc8d-43b4-afcb-343c93605a25 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.940262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.940898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.941176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.941448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.952134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:56.953554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:56.959877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ba382-0f97-4654-ab5b-16a7aa842b65 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:56.969072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:56.970200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:56.971237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:56.971237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:56.971237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:56.971237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 295/570] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:56.971237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:56.996782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:56.997387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:56.997659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:56.997928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:56.998615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:56.998918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.000411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.012053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 276/571] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.012867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.037532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.076547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d502102-0dda-492b-bbcf-4780da729fa0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.111749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.112404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.112679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.112945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.113073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6259d-a226-4adf-b5b8-aac8bd43f55a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.121398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.122840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-11cdd737-c5f1-4b28-b37a-1eb11fa074c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:57.135596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.136730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:57.137859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:57.140147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 296/572] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.141579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ea45f-a747-41b4-85fb-77fdd4c929e4 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.141886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ea45f-a747-41b4-85fb-77fdd4c929e4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.142278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ea45f-a747-41b4-85fb-77fdd4c929e4 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.143572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 297/573] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:57.144368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ea45f-a747-41b4-85fb-77fdd4c929e4 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.152980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.154858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.155133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.155397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.157974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.167530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:57.172314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.173280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:57.174121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 277/574] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.174808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.264135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.267690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ea45f-a747-41b4-85fb-77fdd4c929e4 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.283862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37853f-28e7-494b-a16c-ef06a3a0147a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.307829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.308479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.308761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.309033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.309682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.310226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.312018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.323818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.324453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.324732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.325001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.325760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=dfb941d3781044b0a2092c596bf51e32) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.326070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.326194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 298/575] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.327455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.327575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4be58e-f61e-4c49-9a35-dbb0f7f0bcb4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.344032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 278/576] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/role_assignments?user.id=dfb941d3781044b0a2092c596bf51e32&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.344032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.344032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.344032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.366299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:57.367553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:57.417858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.459503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 279/577] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:57.461298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca76ec-773b-4fad-8c17-2a362cd6a4d4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.461450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.520206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.520739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.521012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.521278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.522140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.522453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.525516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.544316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 299/578] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.544316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.558054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.586691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f7982-8d1d-4bd2-810a-76e4a0884c88 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.628184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.629478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.629942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.630473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.632442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:57.633158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.635883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.646967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 280/579] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:57.648564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.666290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d00b7-fbab-4144-a9a5-aba7be36d9e1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.699237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.707061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.707702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.707976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.712092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.716748 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.718192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:57.733081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.734286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 300/580] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.741770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.788946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3979c9c-1cfd-46d1-a301-01c2b7c590a4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.827927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.828584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.828859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.829126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.838378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:57.844397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:03:57.849216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:57.850221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:57.851448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 281/581] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:57.852937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:57.862729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75206b62-4358-4367-a0d9-a624c7f3a25c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:57.903643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:57.904344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:57.904624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:57.904897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/462638e8386c49668290954c0211728c/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:57.920781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=462638e8386c49668290954c0211728c, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:57.922246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:57.983013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5734e33f-b03e-4ac1-b8ca-2f8e6538ac65 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.011746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 301/582] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:57 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/462638e8386c49668290954c0211728c/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:58.011746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.038100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.038757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.039180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.039493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.052873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.055330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.058503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:58.075154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:58.076542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:58.078305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:58.078305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:58.078305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:58.078305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 282/583] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:58.079332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.154969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.155651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647be5d-4f00-4f0d-8bf7-bfc981237ad0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.207225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.207835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.208148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.208417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.209148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:58.209462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.210968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.220832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82825172-60cd-4330-b9f9-974ad8af32ef None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.231287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 302/584] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.232227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.266914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.267525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.267838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.268149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.268838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:58.269140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.270526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.283750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 283/585] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.285985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.349152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.355733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c6a5cb-d73e-4f20-8312-a6df5c2bc0a9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.398874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.399611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.399890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.400259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.401038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:58.401347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.402722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.415718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 303/586] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.416761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.437675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537d58d-80bb-4312-bd85-3a0ea021f545 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.455048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.483460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.485018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.485590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.486105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.502351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.504868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:58.527260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:58.528460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:58.529531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:58.529531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:58.529531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:58.529734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aba33a-6f09-4986-98f7-b0f2e2e43ad9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.531561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 284/587] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:58.532600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.576630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.577589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.577949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.578285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.593450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.594900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:58.616465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:58.617521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:58.618463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:58.618463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:58.618463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:58.620503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 304/588] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:58.622705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.638006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.654330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5861f-4404-40f9-b9c3-95e292c8d27c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.689421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.690205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.690583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.691112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.699033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.700647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:58.712736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:58.713797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:58.714889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 285/589] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:58.715764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.740268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894a916-4866-4dd8-b737-36b2b5217d3e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.743068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.779164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.780137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.780591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.781047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.781872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:58.782408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.784078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.802171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 305/590] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.803446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.823389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14845bb-7647-4aef-a76d-362cebf4e030 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.859256 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.859956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.860383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.860727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.861482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:58.861853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.863550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.876163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 286/591] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.877046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.879271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.916189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61188058-0b12-400b-a645-d45be53fc53c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:58.954319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:58.955283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:58.955741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:58.956222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:58.957244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:58.957723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:58.959410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:58.964814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:58.980262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 306/592] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:58.981519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:58.981654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a3cba-45c3-45c1-a201-c85b34fd1cbe None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.017797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.018540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.018953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.019335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.026921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.028154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:59.041162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:59.042081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:59.042988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:59.047000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 287/593] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:59.047941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.096567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ec26ee-16c9-4396-8021-04cf95802d7b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.125815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.139852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.139852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.139852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.139852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.146048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.147904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.157681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c067512-afbf-4121-868a-c88f92f52763 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:03:59.161972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:59.169498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 307/594] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:59.170670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.203784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.205637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.206129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.206709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.221929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.223496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.227329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:59.240211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:59.241746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:59.242839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:59.242839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:59.242839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:59.248966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 288/595] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:59.248966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.316132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4ada4-dcf9-47ca-b373-1bd2115245ef None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.351907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.352755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.353211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.353585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.354369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:59.354788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.356352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.361937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5662231-4255-4f7a-8086-e3083b161694 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.376673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 308/596] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:59.377665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.402633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.403581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.404031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.404505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.405375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:59.405848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.407515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.427521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 289/597] 10.222.0.13 () {56 vars in 1139 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:59.429228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.434973 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.491320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b5d4ec-5675-4d65-b87d-9712ecc4ccae None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.492275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.527867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.529005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.529443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.529867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.531115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:03:59.531694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.533510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.539906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965bdfa6-b3ff-4d9c-bf40-f21a2f86d789 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.552164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 309/598] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:03:59.553112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.577910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4caaa9-a345-4448-8366-3b924f4faae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.578561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4caaa9-a345-4448-8366-3b924f4faae4 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.578835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4caaa9-a345-4448-8366-3b924f4faae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.579171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4caaa9-a345-4448-8366-3b924f4faae4 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/3a95ab030c9940c9946c64c82567773c/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.595578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4caaa9-a345-4448-8366-3b924f4faae4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=3a95ab030c9940c9946c64c82567773c, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.596907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4caaa9-a345-4448-8366-3b924f4faae4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.653588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca9674-e049-415c-ac5d-45886a4ccc5c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.660464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.682526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 290/599] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:03:59 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/3a95ab030c9940c9946c64c82567773c/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:03:59.688180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641fb9b-2f26-4ff6-914d-7582fc27969a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.688180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641fb9b-2f26-4ff6-914d-7582fc27969a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.688180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1641fb9b-2f26-4ff6-914d-7582fc27969a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.688180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 291/600] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:03:59.688180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641fb9b-2f26-4ff6-914d-7582fc27969a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.697359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.698016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.698447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.698814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.711873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.713373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-fb2fd79d-2667-4060-9622-ab0a8b2a1ec1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:03:59.734274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:59.735305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:03:59.736233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:03:59.736233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:03:59.736233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:03:59.738228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 310/601] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:59.739690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.740150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.740620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.762212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:03:59.763843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:03:59.830661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1641fb9b-2f26-4ff6-914d-7582fc27969a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:03:59.834444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.845881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 311/602] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:03:59.847029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.893812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:03:59.894448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:03:59.894725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:03:59.894996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:03:59.903293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:03:59.904915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:03:59.918259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:03:59.919182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:03:59.920136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 292/603] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:03:59.920625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:03:59.968587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:03:59.982340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d86a15-2521-4b22-9931-9ccba85b1644 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.018933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.019573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.019852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.020152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.020789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:00.021089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.022970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.037132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 312/604] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:00.037983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.053027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34c8f7-6cd5-4558-a683-fb4d9cf347df None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.090896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.091930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.092456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.092929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.104524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:00.109556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:00.110504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:00.111342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:00.113917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 293/605] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:03:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:00.114992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.145433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.154126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62acc1-f0be-4210-a339-6bf351c8fbdb None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.190609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.191138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.191414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.191681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.192401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:00.192703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.194123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.207477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 313/606] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:00.208321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.221370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.233205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96d872-8dca-4763-bc70-7a8d62cd7752 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.270226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.270858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.271133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.271404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.280184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.281611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-def8c7a7-21b2-4a14-bc2b-ca0395a026e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:00.293834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:00.300673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 294/607] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d893b6-d94a-4ebf-9fbf-01c29229a910 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.301583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d893b6-d94a-4ebf-9fbf-01c29229a910 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.303734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d893b6-d94a-4ebf-9fbf-01c29229a910 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.303734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 295/608] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:00.303734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d893b6-d94a-4ebf-9fbf-01c29229a910 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.318836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6c5233-9a1c-44d4-9222-174a88114b5a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.356831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.357443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.357722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.358000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.358650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:00.358952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.360352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e64dc9-932e-49d5-bf0b-ecff32a17796 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.373029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 314/609] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:00.374296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.374583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.375070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.394411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:00.396497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:00.407335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.424459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d893b6-d94a-4ebf-9fbf-01c29229a910 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.460711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.461615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.461894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.462165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/3cada6c06ef8427395eda888397efac4/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.468696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.482827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 315/610] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:00.482827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.483083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=3cada6c06ef8427395eda888397efac4, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.483083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.579952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc81648-4b9a-4f97-9866-b3250359d93b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.594400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 296/611] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:00 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/3cada6c06ef8427395eda888397efac4/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:00.594400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.634043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.634733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.635043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.635597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.636762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:00.637083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.638910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.656617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 316/612] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:00.657846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.721924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b798e-8f77-4e1d-a6ca-49fdacd1f420 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.739546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.770709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.771652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.771928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.772246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.775445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:00.786569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:00.789508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:00.794160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca7540-af1f-46d5-ab51-b8ad73ff79c9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:00.808227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:00.809222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:00.810118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:00.810118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:00.810118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:00.810740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 297/613] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:00.811152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.835531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.836573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.837078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.837547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.849305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:00.855833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:00.856832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:00.857681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:00.860409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 317/614] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:00.861414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:00.938356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee1dc-5a8b-45bd-8b3a-47c9e04bde84 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.971446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d83b22-8fe2-4da6-b574-b242a797553c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:00.981206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:00.982160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:00.982448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:00.982719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/52773c21d6e9406086113a9fd74d5f62/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:00.994284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.001602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=52773c21d6e9406086113a9fd74d5f62, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.003625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.010734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.011646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.012196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.012650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.013544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.014025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.015681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.030106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 318/615] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:01.031183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.067329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 298/616] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:00 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/52773c21d6e9406086113a9fd74d5f62/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:01.068662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.140834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.196807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fec71-14da-41a5-b4d5-60da9ea6534e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.240282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.242120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.242544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.242954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.243978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.244878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.246911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.259986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 319/617] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:01.266102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.270248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ea4fd-5c97-45de-a9a7-5b0630e1084f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.310074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.310706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.310985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.311265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.319908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.321487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.323150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:01.336190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:01.337629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:01.338790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:01.342673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 299/618] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:01.343588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.429271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2a9bb-bdbc-4b85-a8dc-06f747ac7bd9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.483234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.489190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad79c2-b373-4af9-8e75-a8cbc970955a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.501364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.501943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.502228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.502561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.514272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.521987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.532292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.532911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.533193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.533462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.534195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.534498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.535941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:01.550893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:01.551896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:01.552800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:01.552800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:01.552800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:01.556144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 320/619] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:01.562143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.568928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 300/620] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:01.574215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.681362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cda094-01b8-45ea-8364-7f4f05c58241 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.716606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.717240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.717517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.717786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.728851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.730297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:01.745669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:01.746660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:01.747547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:01.747547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:01.747547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:01.747547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 301/621] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:01.752753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.764351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.769534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:01.790452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da47426-ffee-4c91-881c-3bf3f8e5b636 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.860137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aff9d1-95a7-43a8-90c1-1fcb4c26e411 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:01.892725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.893345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.893623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.893893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.894646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.895031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.896540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.897605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:01.898605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:01.898983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:01.899355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:01.900127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:01.900532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:01.901975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:01.915550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 302/622] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:01.921058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 321/623] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:01.921839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:01.925983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.003251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.046042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dfb9b-1c55-46ef-a283-ad6c2e3c926b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.049618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172c704-5c2d-455b-845f-5992724c6517 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.090687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.091314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.091594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.091862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.093585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.094366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.095148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.095426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.100322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.102788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.103330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.105053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2ee16d44-0deb-40f1-a6ad-7f1e8c52995b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:02.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:02.116072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:02.118212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 322/624] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:02.119200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:02.120785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:02.126310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 303/625] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:02.127252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81974-b997-4ed7-bf35-37dacf8944fd None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.127252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81974-b997-4ed7-bf35-37dacf8944fd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.127252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81974-b997-4ed7-bf35-37dacf8944fd None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.127252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 323/626] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:02.127662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.127793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81974-b997-4ed7-bf35-37dacf8944fd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.141580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.232996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81974-b997-4ed7-bf35-37dacf8944fd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.236105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc917556-e3ea-46bc-89a6-b39569956f5d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.265170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.269047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.269741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.270255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.270255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.271102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:02.271371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.273301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1c936-f94c-41c9-862a-a23d45d4b464 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.275908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.276740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.277162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.277594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.278513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:02.279047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.280843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.290331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 324/627] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:02.290331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.290331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.290331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.306594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:02.307383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 304/628] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:02.307826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:02.308663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.383685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 325/629] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:02.389516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.392915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.433243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9df5cb-9a35-4119-9f65-0dc2b6eea984 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.472017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.472887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.473193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.473511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/dfb941d3781044b0a2092c596bf51e32/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.490123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=dfb941d3781044b0a2092c596bf51e32, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.491748 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.518033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880e5a8-4726-41c8-93b9-21d3f4579af0 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.551529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.552152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.552430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.552699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.617981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:02.619425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:02.718431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 326/630] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:02.718431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.762899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.816818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91d5392-b435-4208-9a0b-d1f47e823427 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:02.832153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 305/631] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:02 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/dfb941d3781044b0a2092c596bf51e32/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:02.833994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:02.862794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.890720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:02.891783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:02.892627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:02.894324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:02.920976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:02.921935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:02.933893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:02.934967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:02.935855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:02.938372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 327/632] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:02.945105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.024527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f82fee3-43f0-40d3-9abc-f40138c9c638 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.065888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4afba-b4d8-4e64-82f4-eee20be037e3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.073412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.073991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.074267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.074536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.085553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.087752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-1bd0b0a5-fe37-4541-b45b-5b686ce03469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:03.106479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:03.107513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:03.108427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:03.108427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:03.108427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:03.108427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 306/633] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:03.109241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af995790-b0f5-4ac9-a277-d9f93603ff68 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.109522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af995790-b0f5-4ac9-a277-d9f93603ff68 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.109790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af995790-b0f5-4ac9-a277-d9f93603ff68 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.111146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.111289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 307/634] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:03.115805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af995790-b0f5-4ac9-a277-d9f93603ff68 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.115945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.115945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.115945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.115945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:03.115945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.120091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e423caec-fdb9-4468-97a7-3562c3f46539 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.131081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 328/635] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:03.133329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.133855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.134449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.154049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:03.155563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:03.180411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.228342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.240084 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 329/636] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:03.246448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.248632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af995790-b0f5-4ac9-a277-d9f93603ff68 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.294995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.295721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.296082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.296438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.298020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:03.298389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.299813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.316564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 308/637] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:03.317513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.374294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9ccf6-fdae-421b-9c2e-36799a3d98f8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.411904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.412923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.414480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.414970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.425499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.426927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:03.445588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:03.446629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:03.447535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:03.447535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:03.447535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:03.447705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafa4e7-4b74-4dd6-b108-4330d12c7fa5 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.447796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 330/638] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:03.448382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.462020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.480549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1e61a-6e64-4d9f-88d5-872d3d8b2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.481314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1e61a-6e64-4d9f-88d5-872d3d8b2f7d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.481654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1e61a-6e64-4d9f-88d5-872d3d8b2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.481939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1e61a-6e64-4d9f-88d5-872d3d8b2f7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.494214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1e61a-6e64-4d9f-88d5-872d3d8b2f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-39b1e61a-6e64-4d9f-88d5-872d3d8b2f7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:03.499342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:03.500410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:03.501255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 309/639] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:03.501967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162a78-4bc8-4b45-b92b-d7e7601438f4 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.501967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162a78-4bc8-4b45-b92b-d7e7601438f4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.502105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8162a78-4bc8-4b45-b92b-d7e7601438f4 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.507468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 310/640] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:03.508247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162a78-4bc8-4b45-b92b-d7e7601438f4 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.517230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.561280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc21cc8-29c8-4081-ba32-3a27daae4404 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.594820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.595450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.595728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.596013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.603408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.604888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.614230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8162a78-4bc8-4b45-b92b-d7e7601438f4 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:03.616793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:03.617883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:03.618778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 331/641] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:03.619297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.658526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.659047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.659321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.659585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.660244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:03.660544 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.661899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3184adae-ebbd-479d-9128-3169a4e4f88f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.675158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 311/642] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:03.676436 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.676717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.676982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.695513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:03.696731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:03.700884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.735043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3fddd2-5e6a-4a15-8b1c-f698af9ee5bf None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.758658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.776027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 312/643] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:03.776027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.778847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.779520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.779857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.780210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.780943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:03.781407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.782836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.792154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 332/644] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:03.793565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.879928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ea0a7-3b16-4392-888b-4f4a5df34d33 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.897520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86e6e8-f5bc-47a9-813f-b4c54836f4ce None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:03.914908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.915454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.915735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.916032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.916719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:03.917025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.918445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.928298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:03.930955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 313/645] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:03.931817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.940036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:03.947459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:03.947459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:03.947459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:03.950199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:03.951676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:03.963155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:03.964701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 333/646] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:03.965814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:03.999973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.049490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e8dd6-8e35-4eeb-a1cd-76be47ddfb7a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.077069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85cb03-fd26-490d-8976-306ebc630aa4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.085458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.086086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.086363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.086638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.097090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.098594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.112032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:04.112822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:04.113730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:04.114615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:04.114615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:04.114615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:04.114615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 314/647] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:04.115252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.115516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.115962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.116940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.128815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:04.135404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:04.141581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 334/648] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:04.142442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.197208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.222782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f602cf9-4a52-4287-8486-19de0b2ca271 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.232430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.249830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3787047-3034-49ef-8782-6c2d5c5cd9ae None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.257924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.258537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.258836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.259078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.259800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=462638e8386c49668290954c0211728c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:04.260142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.261614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.267292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 315/649] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/role_assignments?user.id=462638e8386c49668290954c0211728c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:04.268086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.294613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.295632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.296122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.296716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.297791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:04.298278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.300023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.327535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 335/650] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:04.327535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.412356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa131e2a-5e1e-4311-8935-9abdfc652dca None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.459498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bed25-6aad-443e-980f-e3711137a138 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.459674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.460611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.460889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.461327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.462403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:04.462403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.466373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.471856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.479638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 316/651] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:04.482675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.500511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.501280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.501564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.501836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.502497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:04.502800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.505890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.516649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 336/652] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:04.516649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.526203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.606904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e19c8-5f58-4f9c-8711-ced6bea653df None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.649293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.649987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.650259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.650526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.732879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.735474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.746392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdfa59-9d71-4964-8edf-5f3eb888fe91 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.779522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.783024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.783662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.783938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.784252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.792645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.793742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:04.806068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:04.807062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:04:04.808650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:04.812043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 337/653] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:04.812912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.845750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:04.878936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 317/654] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:04.883309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:04.921480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe79e2-52e8-424d-ad05-0b5c5c56f8ea None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:04.958375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:04.959162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:04.959544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:04.960691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:04.971667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:04.973358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:04.988259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:04.989397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:04.990491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:04.990491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:04.990491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:04.991206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 338/655] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:04.991824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.013903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c13a1-727a-4c04-8b39-16031e27b647 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.045715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.050506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.051107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.051391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.051656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.052405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:05.052707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.054154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0d479-d09a-4aa2-9ccb-ce7a1106dfd0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.076950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 318/656] 10.222.0.13 () {56 vars in 1137 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:05.078531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c0b65-1fc4-4de9-b849-589e4c369214 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.078971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c0b65-1fc4-4de9-b849-589e4c369214 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.079413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c0b65-1fc4-4de9-b849-589e4c369214 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.080857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 319/657] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:05.081692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c0b65-1fc4-4de9-b849-589e4c369214 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.085781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.108854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042262c3-243e-49d8-80a3-61e577210f62 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.149327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.150072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.150436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.150777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.161822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.164737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:05.178265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:05.179699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:05.181365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:05.181365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:05.181365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:05.181365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 339/658] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:05.181365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.214926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c0b65-1fc4-4de9-b849-589e4c369214 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.260067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.260579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.260859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.261131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.261768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:05.262085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.263426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0c9e80-a7b4-427a-9c4c-538edc6c995e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.279495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 320/659] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:05.279495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.279495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.279495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.295244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b7aea-7197-4431-b0c4-b96b1442a185 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.295745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:05.296963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:05.333675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.334278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.334554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.334870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.337723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.373099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.402858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 321/660] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:05.405131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.414930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.417560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.514750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2354a8-96ae-43ac-8c47-d71f5f85ff7d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.530774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 340/661] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:05.531618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.556865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.557898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.558391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.558759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.559599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:05.560013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.561633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.566056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.579398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 322/662] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:05.579398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.618563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.642546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3a897-5f91-465c-9163-6bc258cc903f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.677757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.678389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.679457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.679696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.682687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece556d3-3974-44e1-a7fa-e3b483fd0428 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.687524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.689069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:05.701902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:05.703081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 10 11:04:05.704287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:05.705254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 341/663] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:05.709848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.728887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.729603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.729960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.730298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.742053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:05.747818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:05.748868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:05.749704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 323/664] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:05.750617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.812411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.820846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef69bc0d-24e6-414d-8d57-a7867dfcf257 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.859446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.860111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.860390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.860660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.865853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:05.870261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b7249-dfde-4597-b740-6f31b836c825 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:05.871868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.876759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:05.888833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:05.889807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:05.890721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:05.890721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:05.890721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:05.896428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 342/665] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:05.896428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:05.903590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:05.904236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:05.904497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:05.904763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:05.911599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:05.913594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:05.924956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:05.925901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:05.926793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 324/666] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:05.927854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.015837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb9053-d7ff-4553-b939-89e2f8b6288c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.037496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62652863-cd14-4804-8339-7fb764b03ec4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.052559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.053156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.053431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.054219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.055203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:06.055500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.056953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.076121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 343/667] 10.222.0.13 () {56 vars in 1139 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.076121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.080778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.082511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.083220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.083602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.083971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.084782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:06.085226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.086789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.100065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 325/668] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.100065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.123229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.192250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9014f16d-1c82-44f5-871a-c31c59ba9298 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.208941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8d4024-c648-46a0-bb26-a28a78a28605 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.249883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.250114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.250491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.250769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.251034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.251135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.251519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.251719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:06.251902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.252109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.252976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:06.253448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.253542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.255452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.270331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 326/669] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.270331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.275712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 344/670] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.280133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.346259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.362323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.391569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f19d2-18f4-49c0-9ad1-6ee69b986820 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.392993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb03cc-c69b-4ae9-ad78-eee98c82c62c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.446682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.447296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.447578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.447862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.449809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.450576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.450729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.450970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.463521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.465560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:06.482377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:06.483368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:06.484270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:06.484270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:06.484270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:06.484270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 345/671] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:06.484513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.534044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.535564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.596452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dcbfa-0128-43a2-abd1-74fc716fe09c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.617681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.624163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.636156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.639942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.639942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.639942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.639942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=3a95ab030c9940c9946c64c82567773c) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:06.639942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.639942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.647980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 327/672] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:06.647980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.650581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 346/673] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/role_assignments?user.id=3a95ab030c9940c9946c64c82567773c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.656186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.785645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e88a5-63ec-433d-8c60-30a4508fcf9a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.801118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530149a0-56a1-4016-b872-7b6356727b99 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.826721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.827541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.827922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.828318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.837321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.838939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.842671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:06.843483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:06.843912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:06.844346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:06.845157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:06.845640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:06.847074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:06.854093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:06.855462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:06.856844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:06.860218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 328/674] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:06.864029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.864279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 347/675] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:06.864279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:06.910570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.914987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:06.982910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5692a-1f8f-4f90-b278-ba0cc4031e85 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:06.985886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0850a-f839-449f-9240-264a703faac6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.022033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.023383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.023774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.024176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.025152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:07.025580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.027625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.030261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.031129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.031606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.032125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.043349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.044257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 348/676] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:07.045379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.045641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-14b90b62-bba6-448c-98ef-c2864e29a6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:07.064968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:07.065957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:07.066845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:07.066845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:07.066845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:07.066845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 329/677] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:07.067919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2138a-7994-49cb-aa19-060787060aa0 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.067919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2138a-7994-49cb-aa19-060787060aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.067919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2138a-7994-49cb-aa19-060787060aa0 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.073668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 330/678] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:07.073668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2138a-7994-49cb-aa19-060787060aa0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.162207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.165528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924edfb6-f7d2-415d-b3fc-44d2090bc212 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.177183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.185231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2138a-7994-49cb-aa19-060787060aa0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.201280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.201906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.202326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.202770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] PATH_INFO: `/v3/users/swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.211907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.213575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.224126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.224737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.225017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:07.225212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:07.226383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:07.227809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:07.228532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.228532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=3cada6c06ef8427395eda888397efac4) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:07.228532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.228532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5d5576-28d6-4c66-9c6c-edd0c7927806 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.231117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 349/679] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:07.231841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.240302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 331/680] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/role_assignments?user.id=3cada6c06ef8427395eda888397efac4&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:07.240302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.240302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.240302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.263057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:07.264342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:07.357555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 332/681] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:07.358206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d754e-44fe-4279-8aa9-d6752c9b76ba None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.358464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.408351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.409268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.409693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.410126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.410983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:07.411434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.413644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.428545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 350/682] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:07.430238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.457308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.489999 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.506290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9843eb-7c11-4bd0-8763-ed606537e60f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.552423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.553009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.553293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.554191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.555450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:07.555762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.557246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.576848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 333/683] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:07.577792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.579975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58ac55-c055-44d2-86c0-27e739e5a327 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.637340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.638134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.638417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.639051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.656618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:07.667893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:07.671895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 351/684] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:07.676585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.709174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4efcd-4496-4e0c-8ade-f8ab3f449713 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.755517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.756421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.756773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.757253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.765961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.768104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.769938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:07.784368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:07.792372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 334/685] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:07.793458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:07.796777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:07.800870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8994ce-9791-4b15-b896-b2cc32c624fa None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.844582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.845465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.845807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.846083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.909693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccf2c0-33ca-485d-a24e-ebe4ca6948fa None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:07.927401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.929370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.957572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:07.959091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:07.959954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:07.961003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:07.962541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:07.963192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:07.965634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:07.978601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 335/686] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:07.979895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.034010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:08.061021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 352/687] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:08.061021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.073414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:08.118221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269eb37-65c2-449a-a9f3-c3cf0aaa6b0b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.164282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.165179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.165610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.166028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.166435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e00059-9018-4521-9d87-fc8fc9661018 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.167025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:08.167493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.170061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.186528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 336/688] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:08.191921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.215067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.216034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.216462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.216880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.232553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.234186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:08.255298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:08.256520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:08.258521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:08.258521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:08.258521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:08.260921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 353/689] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:08.262027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.305372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:08.308275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f6d94-992e-4e2c-9e4d-64c8f199d32b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.342376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:08.351354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.352171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.352512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.352843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.363790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.367499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:08.390644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:08.391849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:08.392978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:08.392978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:08.392978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:08.399339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 337/690] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:08.399339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.413161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c64c8-ecb8-43e0-9e14-effeaff9fbf0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.461945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.463406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.463836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.464289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.465805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=52773c21d6e9406086113a9fd74d5f62) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:08.466286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.468128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.484820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 354/691] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/role_assignments?user.id=52773c21d6e9406086113a9fd74d5f62&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:08.485148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.537386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b00107-1dad-45dd-80ca-79eb31a7f63e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.614892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.615871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.617014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.617597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.618562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:08.619092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.620807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.628914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:08.636667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c128-6358-42c0-95f8-bfd411e3f91c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.655589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 338/692] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:08.663745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.695108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.697114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.697660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.698333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.700843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:08.701458 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.704255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.729053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 355/693] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:08.734456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.748102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:08.829236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35c4cc-8c91-4a8e-b061-131308250477 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.867453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.868118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.868396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.868667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.876563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.876999 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1395236-d0f6-4667-a4cd-0987b053d61b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:08.877985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:08.890479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:08.891531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:08.892567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 339/694] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:08.898047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.921958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:08.922867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.923147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.923419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.933860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:08.935312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-3521f877-d8af-49bc-8a4d-8c1700581fdf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:08.947595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:08.948613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:08.949492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 356/695] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:08.950447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b28ec-840d-4fb5-b968-b4a0dcc6519b None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:08.950554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b28ec-840d-4fb5-b968-b4a0dcc6519b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:08.951982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b28ec-840d-4fb5-b968-b4a0dcc6519b None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:08.955819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 357/696] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:08.955819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b28ec-840d-4fb5-b968-b4a0dcc6519b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:08.977786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.021091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.026171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808862b-70af-4a11-95e0-09c18292128a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.078289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b28ec-840d-4fb5-b968-b4a0dcc6519b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.086861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.087899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.088446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.088952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.090771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:09.091316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.094096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0656ddb9-01d3-4ef8-9d2e-de496289a66a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.123696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 340/697] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:09.124873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.125685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.126162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.126278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.126661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.126857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.127481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.127835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:09.128321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.129991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.145575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 358/698] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:09.146712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.160762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:09.162671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:09.264057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 341/699] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:09.264057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.275474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.312925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44504a-7fcd-4b4f-810c-3c1e83195809 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.350333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.351677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.352187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.352563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.392430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.408678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b627f6d-a99e-4351-9df7-1b68519cff2a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.446583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.448556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.463053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.464328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.465018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.465423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.466579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=dfb941d3781044b0a2092c596bf51e32) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:09.467480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.469353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.479000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 342/700] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/role_assignments?user.id=dfb941d3781044b0a2092c596bf51e32&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:09.480038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.598629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.664223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 359/701] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:09.669938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.789596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cd32-7740-4658-b010-975e4a9d12f8 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.823247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a02a4bc-43a2-4c5c-a4ca-e66d0ac22d07 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:09.834140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.834584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.835403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.835761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.849678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.851879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a6cd6ecb-1166-4213-bdcc-761529b1bbf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:09.868981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:09.870046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:09.870947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:09.870947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:09.870947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:09.870947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 343/702] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:09.872655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3105f7-68ec-49de-a731-06903ba77e47 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.872655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3105f7-68ec-49de-a731-06903ba77e47 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.872655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3105f7-68ec-49de-a731-06903ba77e47 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.873525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:09.874394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.874782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 344/703] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:09.875280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.875718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3105f7-68ec-49de-a731-06903ba77e47 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:09.875988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.886221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:09.903877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:09.906776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-39df5b7d-d805-4382-90f9-bfc0b1e429cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:09.929738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:09.931043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:09.933702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:09.933702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:09.933702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:09.933702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 360/704] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:09.939699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:09.939699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:09.939699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:09.962710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:09.964029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:09.992276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:10.029536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3105f7-68ec-49de-a731-06903ba77e47 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.060411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 361/705] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:10.061522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.086575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.087248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.087524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.087792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.088478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:10.088774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.090145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.111255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 345/706] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:10.112168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.203423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a81f9-6382-4f42-8688-e5ed5c607137 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.206885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:10.253359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.254364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.254828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.255425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.256441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:10.257211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.259351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.264160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbd5e5c-e073-465e-9962-f457010d8f46 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.282073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 362/707] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:10.283488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.312175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:10.317864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.318987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.319374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.319865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.320563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.322123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:10.334726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:10.336633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:10.338255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:10.344138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 346/708] 10.222.0.13 () {56 vars in 1129 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:10.345161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.441042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1731f15-f172-4a49-afd0-5733856d7ba7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.479221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.480105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.480478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.480944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.481672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.483116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.484043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9016e26-b745-441b-a17a-3521fafed0a3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.514211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:10.522156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.523774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.524228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.524632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.531139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.532650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:10.544097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:10.544991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:10.545886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 347/709] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:10.546361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.580334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 363/710] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:04:10 2022] POST /identity/v3/users => generated 254 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:10.581783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.729868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad43fe-39de-4df0-9324-b63006b06a26 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.751000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:10.785110 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.785876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.786710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.787081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.787817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:10.788895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.791098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.798375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 348/711] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:10.806223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.814168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20769b3-fb43-4d25-bab7-9f085a5fc726 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.856095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.856779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.857057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.857327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.858026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:10.858378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.859815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.869206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:10.874400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 364/712] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:10.880990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:10.929535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca2642-be7a-4184-971e-95463f3eafc9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:10.974059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:10.974673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:10.974950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:10.976369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:10.977019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:10.977322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:10.979095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a23d4cb-0b29-469e-b3e0-029ffd726dac None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:10.988101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 349/713] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:11.016241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2ed221-4bb0-4e56-83d2-1419f256d5cc None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:11.074000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:11.074610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.074884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.075149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.075958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=462638e8386c49668290954c0211728c) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:11.076289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:11.077848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7aede-e1bb-4d44-a781-f71098e67d38 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:11.079785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:11.086961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 365/714] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:10 2022] GET /identity/v3/role_assignments?user.id=462638e8386c49668290954c0211728c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:11.389455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bb565-b365-469d-8f94-49df6b2cde0b None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.389969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bb565-b365-469d-8f94-49df6b2cde0b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.389969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96bb565-b365-469d-8f94-49df6b2cde0b None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.391321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 350/715] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:11.396810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.397871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.397871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.419270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:11.420494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:11.503469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 366/716] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:11.537121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bb565-b365-469d-8f94-49df6b2cde0b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:11.690239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96bb565-b365-469d-8f94-49df6b2cde0b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:11.709449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:11.732964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:11.733596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.733871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.734137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] PATH_INFO: `/v3/services/network` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.734716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:11.738190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-8a3801f1-4e7d-4b86-a83f-425f17f8cf48 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:11.751712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:11.754328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:11.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 351/717] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:11 2022] GET /identity/v3/services/network => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:11.761489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:11.902891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cb7ed-6db5-4a9c-a9c4-a5c88661006d None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.903290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cb7ed-6db5-4a9c-a9c4-a5c88661006d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.903468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424cb7ed-6db5-4a9c-a9c4-a5c88661006d None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.905019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 352/718] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:11.911282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d7a58-6a55-4d48-99e4-c1cf2b00a6cc None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:11.914473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.914752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.915024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.945167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:11.950249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:11.964194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:11.976985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fc46e-317b-493f-bf92-73712036518a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:11.977871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fc46e-317b-493f-bf92-73712036518a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:11.978311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fc46e-317b-493f-bf92-73712036518a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:11.979308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fc46e-317b-493f-bf92-73712036518a None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:11.979414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fc46e-317b-493f-bf92-73712036518a None admin] RBAC: Adding query filter params (name=network) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:11.979645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fc46e-317b-493f-bf92-73712036518a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:11.981017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fc46e-317b-493f-bf92-73712036518a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:11.988436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 367/719] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:11 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:11.995399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fc46e-317b-493f-bf92-73712036518a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.092128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 353/720] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:12.148455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.165278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fc46e-317b-493f-bf92-73712036518a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.246802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.247417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.247801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.248097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.248727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] RBAC: Adding query filter params (type=network) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:12.249089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:12.250711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.256463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32829fdf-6b0d-43e2-881c-09c891bcd012 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.262160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 368/721] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:12.273686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:12.321770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.322419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.322696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.322968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.345183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:12.350100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:12.351421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:12.352413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 354/722] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:12.357413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.422806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:12.483405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87404a30-3715-4e6a-bc46-8b303d45ad80 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.523829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.524380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.524659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.524926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.525633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:12.525934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:12.527416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.539145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 369/723] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:12.545166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.673529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22657668-dee9-400d-888d-f7716f59f7ef None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.719521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.720488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.720971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.721417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.733341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:12.735066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:12.754128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:12.755108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:12.760341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:12.760341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:12.760341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:12.762218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 355/724] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:12.767931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:12.911808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586dc57-cbba-4624-85ca-4d0fe48ac8de None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:12.940287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:12.964156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:12.964793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:12.965129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:12.965406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:12.966071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:12.966428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:12.967799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:12.985174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 370/725] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:12.990701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:13.037636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:13.149895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b115b-345c-49bc-adc7-a95f02967aa3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:13.210724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:13.211279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.211553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.211817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.224727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:13.226506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-197c7729-e333-4752-ad0a-0dc4332547d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:13.241199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:13.242446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:13.243433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 356/726] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:13.249133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:13.279948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1160a87-30ba-4f9d-a4a4-e06fc579e6fe None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.281607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1160a87-30ba-4f9d-a4a4-e06fc579e6fe None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.281879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1160a87-30ba-4f9d-a4a4-e06fc579e6fe None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.284471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 357/727] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:13.289459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.289604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.289839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.321324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:13.322513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:13.442652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a3103d-e71e-44d5-a9fe-50fc2c5ea06f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:13.455042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 358/728] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:13.483682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:13.484241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.484583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.484854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.485650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:13.485955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:13.487429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:13.501027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:13.512692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 371/729] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 10 11:04:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:13.520150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:13.524903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:13.560185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:13.627951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabcd5e5-4214-4e4a-816d-c35852f18f36 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:13.669187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13a722-60d9-4d46-a16c-e3ff978932d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:13.669806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a722-60d9-4d46-a16c-e3ff978932d9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.670085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a722-60d9-4d46-a16c-e3ff978932d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.670357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13a722-60d9-4d46-a16c-e3ff978932d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.709187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a2bbd-29f4-44bb-ae90-d8d0075576c7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:13.762549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7272c63-6b6c-4498-9d8a-de0180aa0306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:13.763520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7272c63-6b6c-4498-9d8a-de0180aa0306 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.764147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7272c63-6b6c-4498-9d8a-de0180aa0306 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.764657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7272c63-6b6c-4498-9d8a-de0180aa0306 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.771266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13a722-60d9-4d46-a16c-e3ff978932d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:13.773357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13a722-60d9-4d46-a16c-e3ff978932d9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:13.777470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7272c63-6b6c-4498-9d8a-de0180aa0306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a7272c63-6b6c-4498-9d8a-de0180aa0306 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:13.782798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:13.783712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:13.784878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:13.787381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 359/730] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:13.789649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd42af-c575-456b-9730-adf4c8910cce None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.790144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd42af-c575-456b-9730-adf4c8910cce None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.790612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd42af-c575-456b-9730-adf4c8910cce None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.796615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 360/731] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:13.797787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd42af-c575-456b-9730-adf4c8910cce None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:13.808290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:13.885450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 372/732] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:13.886790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.887329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.887671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.889609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:13.913119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:13.914487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:13.922076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd42af-c575-456b-9730-adf4c8910cce None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:13.959386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:13.961187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:13.961775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:13.962265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:13.963184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:13.963728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:13.965391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:13.974927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 361/733] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:13.976350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:13.999877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 373/734] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:14.001005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.077375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.089532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c64fb-a0c5-46fa-b6b1-4ac6a51c8054 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.113069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173558e9-4b4c-47b1-b992-bdd6e303c53d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.124105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.124735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.125084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.125358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.127264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.136933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.138348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.148315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.148938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.149214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.149514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b3191e79-7ad1-426c-8309-28a35e81d29e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:14.152732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:14.153675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:14.154568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:14.154568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:14.154568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:14.154568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 362/735] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:14.155304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0fb83-3e31-4dd2-94d1-50d91e6e5edd None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.155583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0fb83-3e31-4dd2-94d1-50d91e6e5edd None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.155851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0fb83-3e31-4dd2-94d1-50d91e6e5edd None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.157278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 363/736] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:14.157838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0fb83-3e31-4dd2-94d1-50d91e6e5edd None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.168905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.170337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:14.185483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:14.186859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:14.187949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:14.187949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:14.187949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:14.191247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 374/737] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:14.192116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.269485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0fb83-3e31-4dd2-94d1-50d91e6e5edd None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.302457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8a0be-0549-4463-932e-50efe00777c4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.306579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983eb214-12b4-4ae4-8097-4d0931ed43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.307317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983eb214-12b4-4ae4-8097-4d0931ed43c7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.307607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983eb214-12b4-4ae4-8097-4d0931ed43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.307877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983eb214-12b4-4ae4-8097-4d0931ed43c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.319054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983eb214-12b4-4ae4-8097-4d0931ed43c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:14.320687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-983eb214-12b4-4ae4-8097-4d0931ed43c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:14.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:14.325325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:14.326195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 364/738] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:14.327521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.327521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.327521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.343570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.344211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.344493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.344767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.345404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:14.345546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:14.345702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.346775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:14.347064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.364523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 375/739] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:14.364523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.400087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.424550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 365/740] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:14.425175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.486058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7840-a1bf-424e-9e4d-69c460aeb9e8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.519617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.520392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.520801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.521148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.521876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:14.522816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.524465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.533802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b58383-2ae0-410c-a5d9-1b9719bf002e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.535598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 376/741] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:14.541719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.569292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.570056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.570331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.570601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.571254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:14.571557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.573030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.581093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 366/742] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:14.581874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.586286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.623813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.647890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f20d-1e75-4e76-a6b6-3423faa90a95 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.681889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.682495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.682773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.683045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.683263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115f174-d8bb-4451-84d8-43f42f075f66 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.689560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.690986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:14.702820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:14.703854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 377/743] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:14.710440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.718474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.719076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.719362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.719624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.720290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.721639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.798794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.802521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b4499-6d42-4571-9fd9-4e046a383e03 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.808414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 367/744] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:14 2022] POST /identity/v3/services => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:14.809278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.847161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.847813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.848185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.848555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.857075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:14.858545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-af02e426-71a8-465d-90eb-0ab38ef15f03 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:14.871956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:14.873054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:14.873985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 378/745] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:14.875267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9c00c-b1e7-4b89-b614-a20c22a94dce None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.875267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9c00c-b1e7-4b89-b614-a20c22a94dce None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.875267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9c00c-b1e7-4b89-b614-a20c22a94dce None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.880903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 379/746] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:14.880903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9c00c-b1e7-4b89-b614-a20c22a94dce None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:14.946352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:14.951026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcb20e0-6ce4-4577-8e73-9c24da51700c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:14.990765 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:14.991454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:14.991733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:14.992028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:14.999276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9c00c-b1e7-4b89-b614-a20c22a94dce None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.004043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.005900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:15.020604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:15.021786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:15.023460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:15.023460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:15.023460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:15.023460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 368/747] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:15.023460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.036399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.037005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.037281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.037554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.039193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:15.039501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.040952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cb278-e831-4b79-8642-94aff50d18c8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.053161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 380/748] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:15.054236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.054518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.054787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.072494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:15.073680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:15.083575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:15.137711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792acb8-6c6d-4276-bf11-90c67f3e0b2d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.144392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 381/749] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:15 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 10 11:04:15.145996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.179068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.179770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.180401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.180740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.181470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:15.182033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.183189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.201652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 369/750] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:15.202315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.213919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:15.264877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd932f0-8820-41a1-969d-7524b573d5df None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.304586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.306975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.306975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.306975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.307249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b21246-a582-4980-a356-c4bab266b12b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.307328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:15.307328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.308583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.323220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 382/751] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:15.324055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.328393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:15.348924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.349631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.349824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.350071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.351633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=dfb941d3781044b0a2092c596bf51e32) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:15.351940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.353384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.368368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 370/752] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/role_assignments?user.id=dfb941d3781044b0a2092c596bf51e32&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:15.368368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.455741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f721e-7e7b-4fba-8758-5b67dbbf5fad None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.470370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:15.497202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.499445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.500561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.501002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.501994 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.505402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.509900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2a0f0-bea3-40aa-ab59-c5ce31c8a7d4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.552925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.554261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.555333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.555608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.612011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 383/753] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:15 2022] POST /identity/v3/services => generated 240 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:15.613641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.657237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.658743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.703665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:15.765251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5970b2-50ef-4655-a3f4-1c756fd4813c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:15.811366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:15.812504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:15.812981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:15.814196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:15.824344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:15.826354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:15.828944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:15.843981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:15.845026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:15.845965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:15.854867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 384/754] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:15.856567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.910290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 371/755] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 10 11:04:15.917003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:15.991904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cadb8f4-f416-44aa-9666-a33694a23cd3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.014889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:16.032561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.033413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.033833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.034253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.035111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:16.035552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.037148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.046777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64945906-545b-4b09-a43e-acc4a6dd6d6a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.066441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 385/756] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:16.067879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.089022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.089784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.090286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.090521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.104576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.105943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:16.122231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:16.123493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:16.124585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:16.124585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:16.124585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:16.125858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 372/757] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:16.132101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.159261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:16.186796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9afc4a-883a-4f36-950b-d3b1b471b3e5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.226017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.226920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.227344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.227771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.238754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.239396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a6b0-1754-4e94-9dbd-362d60d903d9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.240811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:16.258938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:16.260144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:16.261265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:16.261265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:16.261265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:16.267207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 386/758] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:16.268721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.278177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.278903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.279294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.279656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.280422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:16.280827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.282270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.294541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 373/759] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:16.299960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.306902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:16.390234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559c82f-2695-41e5-8b45-1f4170520171 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.396022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:16.414107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab6d67-16d2-49a6-b0e2-b1eb93c6570d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.431020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.431840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.432330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.432997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.433974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:16.434766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.436493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.456174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 387/760] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:16.457718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.504728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.505615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.505890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.506254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.525108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.526719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:16.554198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:16.555609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:16.556695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 374/761] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:16.561851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.568527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:16.581885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c39d-eb1d-4922-851a-b4ff047e7a24 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.622558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.623194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.623523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.623801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.632394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.634200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:16.646733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:16.648211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:16.649103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:16.654065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 388/762] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:16.655258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.771734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3ac1-1b02-4798-ae46-a3e3e1121d2b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.809464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.810033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.810310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.810576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.811256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:16.811558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.813065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.832033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 389/763] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:16.833267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.846669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:16.851473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261d5dc-7297-4d01-8b96-2109cd8a9630 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:16.897532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:16.898298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:16.898627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:16.898900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:16.899641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:16.899950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:16.901844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:16.922189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 375/764] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:16.928166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:16.933605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:17.039860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea7ca7-361a-4596-994d-401c5d250934 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:17.080458 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:17.081123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.081403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.081731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/462638e8386c49668290954c0211728c/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.083699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=462638e8386c49668290954c0211728c, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:17.085332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:17.088516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b194c-11e0-47dd-a0a1-5e0de7f6bd57 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:17.123525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:17.124176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.124462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.124736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.125454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=832be98ac39748009caa1124bf1b3a4c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:17.125766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:17.127190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992523a-eb28-4677-9392-47c9a43d8901 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:17.137237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 376/765] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:16 2022] GET /identity/v3/role_assignments?user.id=832be98ac39748009caa1124bf1b3a4c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:17.137237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ff6e-01cf-451e-afa7-412416e6f07f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.137237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ff6e-01cf-451e-afa7-412416e6f07f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.137237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae40ff6e-01cf-451e-afa7-412416e6f07f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.140403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 377/766] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:17.141850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.142122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.142394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.162438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:17.164406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:17.190103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 390/767] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:16 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/462638e8386c49668290954c0211728c/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:17.249720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:17.259187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 378/768] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:17.280352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:17.396401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:17.432037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7229c-c102-44b9-a207-39335d7666be None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:17.470623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:17.471826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.472206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.472556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.473214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:17.474651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-eef90bf5-f763-4904-a83e-9cc4517305ae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:17.486549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:17.487801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:17.488867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:17.491228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 391/769] 10.222.0.13 () {56 vars in 1129 bytes} [Wed Aug 10 11:04:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:17.496400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:17.633859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda5522-d593-4ec9-8f0f-c833a6e06a8e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:17.669724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f72215-b253-4b91-843e-c010f28d89a2 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.670233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f72215-b253-4b91-843e-c010f28d89a2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.670348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f72215-b253-4b91-843e-c010f28d89a2 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.671795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 392/770] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:17.676915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.677192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.677460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.687860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:17.688494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.688768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.689034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.689662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:17.689959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:17.691438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:17.700046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 379/771] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 10 11:04:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:17.701855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:17.706039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:17.707423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:17.780447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:17.808334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255196d0-c960-4c60-9d7b-8da91cc2ee48 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:17.849626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:17.851772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:17.852716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:17.853144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:17.853633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 393/772] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:17.854090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:17.854533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:17.856061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:17.861871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 380/773] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 10 11:04:17 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:17.867295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:17.879054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:17.937589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:17.976445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c99d37-e467-4ac9-bcfa-ede9c49dde2a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.012977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:18.013600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.013880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.014151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.015098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] RBAC: Adding query filter params (interface=public, service_id=bd40340fad1d4f7e8ee96d256cb4ba90, region_id=RegionOne) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:18.015328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:18.016739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:18.025807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 394/774] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:17 2022] GET /identity/v3/endpoints?service_id=bd40340fad1d4f7e8ee96d256cb4ba90&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:18.032869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:18.113242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b0efe-7141-4232-940d-e4701541ed40 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.175327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:18.188045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafcb0a-1776-49b1-8ebc-c10b863e6e57 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.201620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:18.202266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.202546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.202818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.216675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:18.225569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:18.226162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.226452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.226722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.227324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:18.228708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:18.230776 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:18.234226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 395/775] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:18 2022] GET /identity/v3/services => generated 772 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:18.237139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:18.238204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:18.239810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:18.246496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 381/776] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:18.246664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:18.451004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7313f-3031-4bd7-ad1a-1b820abb6e65 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.488411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:18.489020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.489298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.489573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.490233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:18.490534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:18.492688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ddcd0-5cfa-4de2-a464-034c4e288d97 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:18.503156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 396/777] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:18.508922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:18.600756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34364549-4497-45de-8225-422cfbfbeec9 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.601064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34364549-4497-45de-8225-422cfbfbeec9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.601688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34364549-4497-45de-8225-422cfbfbeec9 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.602743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 397/778] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:18.608892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.609169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.609438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.642641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd788e99-beea-4cdd-bf25-af836951f86d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.644157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:18.645439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:18.683065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:18.684051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.684523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.685016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.707031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:18.709148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:18.711071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:18.719368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 398/779] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:18.731163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:18.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:18.733511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:18.733511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:18.733511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:18.735614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 382/780] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:18.741983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:18.750801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:18.822575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:18.925937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73df9-4b37-4fc3-839a-5188e352fafc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.946181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffc40c-5b9f-48b5-a8b0-515be7db126a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:18.969392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:18.970295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:18.970759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:18.971212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:18.971991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:18.973958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:18.985580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:18.986606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:18.987442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:18.990265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 383/781] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:18 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:18.999531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.037330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.038519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.038831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.039069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.039715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:19.044077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.044077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.055952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 399/782] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:19.066569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.156366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:19.186034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5cb249-2b3a-4f86-bee0-8f81019e31c1 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.190036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:19.228255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.228783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.229062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.229331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.229960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] RBAC: Adding query filter params (name=network) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:19.230278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.231689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.236679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 384/783] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:19.241876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.323734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16190fb0-e21b-44a5-a714-491878bcde6b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.366066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8dd58-f418-40f7-b2c9-2eac62ea5063 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.400303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.400937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.401216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.401486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.414280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.415721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:19.433026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:19.433976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:19.434898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:19.435422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 400/784] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:19.445284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.445702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.445984 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.446258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.446901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] RBAC: Adding query filter params (type=network) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:19.448230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.448577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.450468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.463109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 385/785] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:19.469668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.520564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:19.626689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aa180-9347-41b2-9f52-ce167e4b10d6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.646368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:19.658914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2d731f-f53f-4cd3-a91e-5761042852cb None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.668483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.669104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.669381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.669652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.670381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:19.671039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.672838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a299963-d33d-44e1-ae6e-f82d216b8dce None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.700115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 401/786] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:19.700115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878ae69-5a8f-4bcf-8d13-eab8ca588cf4 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.700115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878ae69-5a8f-4bcf-8d13-eab8ca588cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.700115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878ae69-5a8f-4bcf-8d13-eab8ca588cf4 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.700115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 402/787] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:19.705534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878ae69-5a8f-4bcf-8d13-eab8ca588cf4 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.708400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.709036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.709312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.709663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.711107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] RBAC: Adding query filter params (interface=public, service_id=791ff64a98a04eacac5fcb42b8b347ec, region_id=RegionOne) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:19.711420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.712838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afade0-4809-41ef-845e-f08d9731880e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.722260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 386/788] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/endpoints?service_id=791ff64a98a04eacac5fcb42b8b347ec&interface=public®ion_id=RegionOne => generated 194 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:19.723467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feef53-6b03-40bb-bb17-064a29b70cdb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.723800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feef53-6b03-40bb-bb17-064a29b70cdb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.724159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00feef53-6b03-40bb-bb17-064a29b70cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.744504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-00feef53-6b03-40bb-bb17-064a29b70cdb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:19.745696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-00feef53-6b03-40bb-bb17-064a29b70cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:19.816444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878ae69-5a8f-4bcf-8d13-eab8ca588cf4 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.816971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 387/789] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:19.817538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b12d28-d404-4260-a6d6-e0ec43b2a971 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.817825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b12d28-d404-4260-a6d6-e0ec43b2a971 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.818097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b12d28-d404-4260-a6d6-e0ec43b2a971 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.820064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 388/790] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:19.824583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b12d28-d404-4260-a6d6-e0ec43b2a971 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.854182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:19.861304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.861932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.862276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.862475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.873103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.874727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-faf7f734-913c-4c69-9b9c-5b91cb5b3d2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:19.889810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:19.890840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:19.891729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:19.891729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:19.891729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:19.891729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 403/791] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:19.892461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.892735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.893000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.910429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:19.911604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:19.922103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:19.945181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b12d28-d404-4260-a6d6-e0ec43b2a971 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:19.980427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:19.981041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:19.992016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:19.992174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 404/792] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:19.992270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:19.992270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:19.992421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:19.992503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:19.997355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 389/793] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:19.998215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.093404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.106897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697788a4-15f5-40f5-9999-3dd47f3b2e40 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.111760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e416b-44dc-40eb-9156-ae0478932935 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.146099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.147045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.147484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.147913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.148811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:20.149320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.150840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.157941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.159400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.159753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.160167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.165757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 390/794] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:20.166877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.181800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:20.188906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:20.189952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:20.190796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:20.196137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 405/795] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:20.197187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.205621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.297227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e002d7f-c697-4514-8c55-87c3c700b56b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.344840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.345169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc1d0a-edc6-4293-bc1d-5f08fa3578e6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.345897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.346338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.346956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.366939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:20.374762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:20.375916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:20.376820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:20.377944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 391/796] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:20.378520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.385523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.386761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.387243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.388362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.395966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.397247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.397547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:20.409533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:20.413164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 406/797] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:20.414060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.489029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.519300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2d1dd-8e5f-4700-a595-2cfe01b26082 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.536468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e065b93-be9b-4edf-b210-77efbc798877 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.562807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.563652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.564128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.564558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.565386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:20.565885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.567386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.577045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 392/798] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:20.578331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.587590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.588494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.589018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.589467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.590306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:20.590807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.592450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.602772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 407/799] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:20.604023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.663685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.700959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9c4bd-0d5a-45e9-8198-4ab8a96ba254 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.718343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d748fd-e16f-404a-a835-748a9bf3124d None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.737486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.738095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.738374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.738762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.739514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:20.739826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.741871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.745277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.754325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.754961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.755236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.755509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.760373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 393/800] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:20.767281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.767403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.767507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b1b9ea6d-b408-4760-9452-590b15b3985c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:20.783082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:20.784248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:20.785350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:20.785350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:20.785350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:20.786724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 408/801] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:20.788739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0aaa2-c98a-4088-bbf5-30208a4b83ff None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.789027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0aaa2-c98a-4088-bbf5-30208a4b83ff None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.789299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0aaa2-c98a-4088-bbf5-30208a4b83ff None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.794486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 409/802] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:20.794486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0aaa2-c98a-4088-bbf5-30208a4b83ff None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.874994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da84706-719b-4596-bd4e-43915e5f7348 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.902607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0aaa2-c98a-4088-bbf5-30208a4b83ff None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:20.903434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:20.908786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.909411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.909687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.909957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.920206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.921671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:20.937325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:20.938244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:20.939126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:20.939126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:20.939126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:20.939126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 394/803] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:20.939347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:20.947367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaab240-ddfc-431b-a2bf-3a84f2d14dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:20.948282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaab240-ddfc-431b-a2bf-3a84f2d14dd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:20.948740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaab240-ddfc-431b-a2bf-3a84f2d14dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:20.949196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaab240-ddfc-431b-a2bf-3a84f2d14dd0 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/dfb941d3781044b0a2092c596bf51e32/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:20.951276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaab240-ddfc-431b-a2bf-3a84f2d14dd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=dfb941d3781044b0a2092c596bf51e32, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:20.952734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaab240-ddfc-431b-a2bf-3a84f2d14dd0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.016392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 410/804] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:20 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/dfb941d3781044b0a2092c596bf51e32/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:21.017597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.017877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.018147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.039739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:21.041016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:21.053914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd095-ecc0-4a8c-982e-e49b88cea0e5 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.081069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:21.097814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.098832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.099307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.099773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.101312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:21.101696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.103178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.123246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 395/805] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:21.124242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.140588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 411/806] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:21.142851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.190424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:21.263472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67eaa57-5aef-42a7-9c22-786b82e34ad1 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.289217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cac9ef7-f515-48aa-92bc-c839199a7f3f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.308953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.309609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.309885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.310147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.310788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:21.311083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.312543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.325567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 396/807] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:21.326426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.338564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.339253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.339538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.339813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.351286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.353589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:21.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:21.369767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:21.370773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:21.374837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 412/808] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:21.376358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.413680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:21.462170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211c273-2644-4c0b-8e52-4ce1cf2196bb None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.464011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:21.499627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17008370-d172-433d-8907-784834d14b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.500459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17008370-d172-433d-8907-784834d14b53 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.500829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17008370-d172-433d-8907-784834d14b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.501189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17008370-d172-433d-8907-784834d14b53 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.501863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17008370-d172-433d-8907-784834d14b53 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.503324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17008370-d172-433d-8907-784834d14b53 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.513143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad76fd2-01cb-41af-b24e-7a9ef62e364b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-17008370-d172-433d-8907-784834d14b53 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:21.514399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:21.515581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 10 11:04:21.516634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:21.518635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 397/809] 10.222.0.13 () {56 vars in 1129 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:21.520049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17008370-d172-433d-8907-784834d14b53 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.557653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.558345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.558621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.558920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.567652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.570329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:21.582840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:21.583837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:21.585420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:21.588295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 413/810] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:21.589206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.666089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17008370-d172-433d-8907-784834d14b53 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.703553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.703885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.704246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.704518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.705256 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:21.705537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.706943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.712460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:21.728676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c9a529-1abf-4fa8-9229-a5e1e64c1a59 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.731946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 398/811] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:21.732760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.763441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:21.775890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.776839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.777313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.777823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.778687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:21.779211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.781329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.789100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 414/812] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:21.790537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.860747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00432587-b2b4-4325-ad2c-fd181d26aae7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.904032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2492dbb-22eb-447f-9491-3e007b8279a5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:21.906957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.908116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.908116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.908413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.909223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:21.909621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.911119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.934971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 399/813] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:21.936316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.948425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:21.949317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:21.950173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:21.950598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:21.966482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:21.968734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:21.989310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:21.990472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:21.991821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:21.991821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:21.991821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:21.993995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 415/814] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:21.994920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:21.998876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.033760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.055922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b2d6b-a532-4390-b892-4c1591c0386d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.094744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.095524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.095980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.096400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.097271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:22.097654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.099117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.105513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 400/815] 10.222.0.13 () {56 vars in 1124 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:22.106451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.117369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1fd6d6-6dbe-42b6-8bd2-a0eb35434160 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.152959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e3729-8198-4478-80fe-79c3a6d76bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.153975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e3729-8198-4478-80fe-79c3a6d76bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.154466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e3729-8198-4478-80fe-79c3a6d76bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.154917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e3729-8198-4478-80fe-79c3a6d76bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.224938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c564c23-1684-4953-bdb8-74d31cd7f69a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.229986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e3729-8198-4478-80fe-79c3a6d76bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.231438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e3729-8198-4478-80fe-79c3a6d76bd2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.234958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.261221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.261822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.262103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.262376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.263049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:22.263355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.264740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.280317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 401/816] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:22.281113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.291869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.343689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 416/817] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:22.345128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e0d27-91d2-4e79-a2dd-ed844ec7ad26 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.345483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e0d27-91d2-4e79-a2dd-ed844ec7ad26 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.345831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e0d27-91d2-4e79-a2dd-ed844ec7ad26 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.347299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 417/818] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:22.347913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e0d27-91d2-4e79-a2dd-ed844ec7ad26 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.401386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013c7aa-74f2-4402-bdcf-c671ddb7c4b6 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.437436 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.438039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.438315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.438583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.439228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.440599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.463808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e0d27-91d2-4e79-a2dd-ed844ec7ad26 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.466232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.498126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.498677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.498989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.499262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.505784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.507230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-e3a1a331-d664-4448-8087-2bf64830366c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:22.518068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:22.518984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:22.519932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 418/819] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:22.520772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.520911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.521143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.539735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:22.548351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 402/820] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:22.548549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:22.548632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.636460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.649148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 419/821] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:22.650053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.705225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b609f4c-1c8b-4d3c-91bd-e60e83e3d902 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.743162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.743802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.744105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.744379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.755193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.756668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.766580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:22.774968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:22.780033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:22.780954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:22.780954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:22.780954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:22.780954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 403/822] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:22.780954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.785557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa24115-b375-4e8e-a1bd-ad5ac3d2b877 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.837202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.838104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.838385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.838656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.839876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:22.840482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.842589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.858684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 420/823] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:22.860079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.900273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da906430-39df-424a-9c98-5d9dc34e8aa2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:22.917935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:22.941756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:22.942393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:22.942669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:22.942938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] PATH_INFO: `/v3/services/network` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:22.943528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:22.945586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:22.959022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:22.960987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:22.961811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 404/824] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:22 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:22.963242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:22.979432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d92d6-5a91-4d47-a5f3-e45468a5aac1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.020897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.021431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.021712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.022095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.023715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:23.024045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.025761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.046793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 421/825] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:23.046793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.058021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.092705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8fcd62-006d-4731-a87a-bd96b08b657b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.132858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b23d3-9697-4f5f-ad3d-cda7d92d313c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.133793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b23d3-9697-4f5f-ad3d-cda7d92d313c None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.134077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b23d3-9697-4f5f-ad3d-cda7d92d313c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.134345 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b23d3-9697-4f5f-ad3d-cda7d92d313c None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/832be98ac39748009caa1124bf1b3a4c/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.153118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b23d3-9697-4f5f-ad3d-cda7d92d313c None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=832be98ac39748009caa1124bf1b3a4c, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.154816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b23d3-9697-4f5f-ad3d-cda7d92d313c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.164660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c7d2a9-bb88-4e22-b96a-ce51efc0f5bb None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.199714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.200524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.200637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.200880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.201503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] RBAC: Adding query filter params (name=network) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:23.201802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.203149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.207441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 422/826] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:23.208225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.209280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 405/827] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:22 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/832be98ac39748009caa1124bf1b3a4c/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:23.210964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374b396-3185-4784-97b5-ec1034b50fd7 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.211428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374b396-3185-4784-97b5-ec1034b50fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.211870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374b396-3185-4784-97b5-ec1034b50fd7 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.213856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 406/828] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:23.214833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374b396-3185-4784-97b5-ec1034b50fd7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.254640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.306439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.333787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63b16d-2568-4af6-b0c2-ece4cca9cc29 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.347006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374b396-3185-4784-97b5-ec1034b50fd7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.380060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.380060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.380060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.380060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.389032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.389210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.389210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.389384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.389458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.389458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.389775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] RBAC: Adding query filter params (type=network) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:23.389932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.391500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5c80c-a373-4d70-acd8-dc2a07e4d80c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.397485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 407/829] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:23.398702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.398702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.398870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:23.402689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:23.403588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:23.404496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:23.407326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 423/830] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:23.410048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.422109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:23.423701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:23.500271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 408/831] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:23.501550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.539150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.557843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6091b-062a-4230-9a53-2098208dd672 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.595852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46413b41-1534-4525-93db-212274f62e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.596952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46413b41-1534-4525-93db-212274f62e93 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.597432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46413b41-1534-4525-93db-212274f62e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.597901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46413b41-1534-4525-93db-212274f62e93 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.598777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46413b41-1534-4525-93db-212274f62e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:23.599301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46413b41-1534-4525-93db-212274f62e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.601232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46413b41-1534-4525-93db-212274f62e93 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.603337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.616532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 424/832] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:23.617717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46413b41-1534-4525-93db-212274f62e93 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.640558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341cece3-cdf6-44be-84f9-d0c19f25a158 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.678216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.679353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.679836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.680590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.681532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:23.683148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:23.739566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46413b41-1534-4525-93db-212274f62e93 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:23.767938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 409/833] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:23.770034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.828467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:23.829183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:23.829459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:23.830509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:23.856644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:23.872439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:23.873435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:23.874268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 425/834] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:23.879983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:23.950741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.959251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:23.979805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d2d857-ebe3-475d-8382-af731c2a002b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.023461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.023669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df3e1a9-b910-4f22-9bbb-77b3138f76c7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.024885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.025511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.026013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.027795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=462638e8386c49668290954c0211728c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:24.028363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.030103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.038269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 410/835] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/role_assignments?user.id=462638e8386c49668290954c0211728c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:24.068091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.068726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.069723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.070008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.070719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:24.071019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.072601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.083040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 426/836] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:24.088698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:24.211816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc30b4c-b14a-4e42-abb7-e5c00caddd7f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.249936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.250576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.250853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.251121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.263087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.264578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.266347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:24.281819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:24.282788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:24.283876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:24.283876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:24.283876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:24.283876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 411/837] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:24.289825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:24.414771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690a65f-a8a6-4526-9905-b89a2211245c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.429377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:24.451759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.452772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.453259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.453754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.454602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:24.455125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.456764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.471145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 427/838] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:24.481107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:24.590052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ab819-e947-4126-b0b2-b90611a9b4e3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.626477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.627450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.627938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.628444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.637075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.638831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-26fe64f6-07cd-41fa-9c2b-64afc99c4779 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:24.650406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:24.651376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 10 11:04:24.652363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:24.654938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 412/839] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:24.660564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:24.689350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120da29-40b8-4fe4-a55c-9fda824a4b9c None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.689679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120da29-40b8-4fe4-a55c-9fda824a4b9c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.689973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120da29-40b8-4fe4-a55c-9fda824a4b9c None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.691481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 413/840] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:24.696861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.697148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.697429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.715624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:24.716847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:24.776303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:24.780779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8850bcf-3a33-484d-bf57-8581ddec40a0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.800468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 414/841] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:24.818463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:24.824149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.824995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.825457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.825939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.826861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:24.827855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.829960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.855380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 428/842] 10.222.0.13 () {56 vars in 1135 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:24.860741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:24.906429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:24.927233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91a344-4bbd-4b34-8067-73b0d5570761 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:24.966586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:24.967353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:24.967712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:24.968092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:24.968752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:24.970185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:24.981034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:24.982031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:24.982898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:24.987204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 415/843] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:24.988188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.004757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67928e31-42a6-488c-a954-94e43afe20c7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.043040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:25.053425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.054006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.054181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.054577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.111484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be58fe-aa07-4f4a-95ec-0674b2561da3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.127423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.129042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.156804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.158012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.158634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.159098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.160268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:25.160976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.162013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:25.162769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.173747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 416/844] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:25.175288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.245795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 429/845] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:24 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:25.255272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.303815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df555a1-c8c2-4193-a246-29566f0e1662 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.317903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:25.343470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.344224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.344731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.345084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.346580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:25.346974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.348505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d85e539-8a5c-40a2-9662-f47f2afd87c4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.354224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 417/846] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:25.392042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f5bf9-8be9-4f2c-ba8f-46b33f5720c6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.421901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:25.426868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cc341-5356-4cbf-989b-46a86fb206c9 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.427255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cc341-5356-4cbf-989b-46a86fb206c9 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.427643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cc341-5356-4cbf-989b-46a86fb206c9 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.429700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.437119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 418/847] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:25.437249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.437249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.437249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.437440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.437440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.437440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.443724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.446444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.456300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:25.457692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:25.464449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:25.465646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:25.467100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:25.467100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:25.467100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:25.467100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 430/848] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:25.472997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.560116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 419/849] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:25.579235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.597590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373aa93-039c-49d4-88ab-20d4eb46cbf8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.648894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.649512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.650042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.650319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.651382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:25.651955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.653421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.669817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 431/850] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:25.676658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.699906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771168cb-aa67-4bdf-8c08-533edbf7fe8b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.708387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:25.737205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:25.738608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.739445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.739902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.740392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.754101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:25.760859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:25.761803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:04:25.762629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:25.765496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 420/851] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:25.772463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.806079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb7ac-cd63-42b4-919b-5246de4e0fe7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.846813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.848169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.848169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.848478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.856696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.858274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:25.871442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:25.872420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:25.873309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:25.875138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 432/852] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:25.880721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.895668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c09b3e-d86b-4400-a3ec-97e5df2a6a15 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:25.931874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:25.932746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:25.933166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:25.933587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:25.934435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:25.934884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:25.936433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:25.945601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 421/853] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:25.950734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:25.988199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.008501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e41496-9f48-4dc7-8787-31be4be90d8e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.010275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.047166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.047775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.048086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.048359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.049043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:26.049397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.050873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.070632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 433/854] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:26.071548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.094208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aeed7f-e2fa-4514-9992-9c62674a3950 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.138826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.140286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.140923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.141449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.154845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.157066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:26.178536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:26.179736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:26.181086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:26.181086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:26.181086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:26.187857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 422/855] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:26.189601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8f017-d16a-4e5c-9a92-125c540c457a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.191864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.238616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.239347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.239698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.240100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.241535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=dfb941d3781044b0a2092c596bf51e32) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:26.241535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.243325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba662d1-d792-407d-8e96-cc722ee42f31 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.252218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 434/856] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/role_assignments?user.id=dfb941d3781044b0a2092c596bf51e32&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:26.252218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8254-2c60-450e-b710-297ea11c0ed2 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.252218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8254-2c60-450e-b710-297ea11c0ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.252218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec8254-2c60-450e-b710-297ea11c0ed2 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.252218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 435/857] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:26.258375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.258667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.258948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.279978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:26.281254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:26.298781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.322804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e59c5f-be9f-4e5f-8903-f68d8be17378 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.365087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.366289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.366752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.366752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.368138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:26.368138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.368769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.369561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.378883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 436/858] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:26.387173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 423/859] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:26.395718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.408508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.523607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afed8b-4a03-431a-a5bf-966aec0ca93b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.537439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0d411-0270-40bc-a1ad-086141c356c3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.560628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.561421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.561769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.562111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.570378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.571895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.581696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.582321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.582601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.582874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.583458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:26.584906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:26.585832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:26.586756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:26.587297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.587383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 437/860] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:26.593098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:26.594438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:26.596050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:26.598141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 424/861] 10.222.0.13 () {56 vars in 1132 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:26.606136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.610186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.619470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.714316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e259d79-29dc-4b16-9608-1bbdfb7c68f5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.738887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e77c1-5a16-43b6-9eb2-240b3486af9f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.753303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.754253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.754399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.754793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.755608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:26.756024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.757937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.780158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 438/862] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:26.782294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.785540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.786333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.786614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.786886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.789239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:26.789559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.790923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.801498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 425/863] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:26.802671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:26.864922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.877601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:26.891888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa1e2f-b72b-430e-b26e-7cac2c47b00c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.912130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0ca28-899c-4ba5-bbc5-36fdde24d12c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:26.927492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.928196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.928520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.928794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.949012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:26.949621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:26.949905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:26.950179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:26.950815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:26.951119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:26.952522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:26.964311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 426/864] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:27.005250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:27.007012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:27.092411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:27.120088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 439/865] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:27.125755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:27.242306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472a4c1-d3f1-4f45-9268-d9438f92df68 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:27.273595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:27.279166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:27.279837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.280164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.280442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.292028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:27.293750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:27.309281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:27.310241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:27.311127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:27.311127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:27.311127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:27.313191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 427/866] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:27.319648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:27.424860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee9dc1-b2e6-4121-ab4e-89157fca2ca6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:27.461972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:27.462596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.462875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.463147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.463790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:27.464123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:27.465496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8548fa9-646f-4027-bea1-39dc35045bf7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:27.479681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 440/867] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:27.485407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:27.487916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67ef83-66ba-437d-8509-6206d6d416b6 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.488243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67ef83-66ba-437d-8509-6206d6d416b6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.488783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67ef83-66ba-437d-8509-6206d6d416b6 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.492025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 441/868] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:27.497049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.497535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.497985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.524382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:27.525842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:27.563116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:27.599858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a541403-e386-4a6e-8f53-e0ceff1707e0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:27.623684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 442/869] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:27.642306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:27.642939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.643216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.643487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.645506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:27.654755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:27.656699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:27.670805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:27.671760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:27.672707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:27.675297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 428/870] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:27.684288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:27.734885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:27.775025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d31c5e-e5b6-4662-aad7-3c5bbb238526 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:27.800039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7f5a1-fb45-4cdb-94ff-2a48352c064b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:27.811972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01badda-858e-4504-a6cb-b0d95464530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:27.812866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01badda-858e-4504-a6cb-b0d95464530e None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.813308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01badda-858e-4504-a6cb-b0d95464530e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.813744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01badda-858e-4504-a6cb-b0d95464530e None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.814543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01badda-858e-4504-a6cb-b0d95464530e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:27.816166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01badda-858e-4504-a6cb-b0d95464530e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:27.838892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:27.839493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:27.839773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:27.840119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:27.840755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:27.841058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:27.842464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:27.855148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 429/871] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:27.860850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:27.893799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 443/872] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:27 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:04:27.984745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7da4ae-1ce5-47bb-a759-7ae25a218c20 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:27.987863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:28.039250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:28.040308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:28.040844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:28.041385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:28.042370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=832be98ac39748009caa1124bf1b3a4c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:28.042914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:28.044578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85536388-14dd-4e78-b3aa-ac7308e882ce None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:28.051710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 430/873] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:27 2022] GET /identity/v3/role_assignments?user.id=832be98ac39748009caa1124bf1b3a4c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:28.422281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8eab7-d472-4a76-8782-7ce38943d825 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:28.423245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8eab7-d472-4a76-8782-7ce38943d825 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:28.423756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8eab7-d472-4a76-8782-7ce38943d825 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:28.425646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 444/874] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:28.431609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:28.432206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:28.432688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:28.452313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:28.453858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:28.524895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 431/875] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:28 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:28.547412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8eab7-d472-4a76-8782-7ce38943d825 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:28.576424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:28.658129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8eab7-d472-4a76-8782-7ce38943d825 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:28.695651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:28.696670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:28.697183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:28.697666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:28.698468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:28.700057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:28.709139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:28.710173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:28.711014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:28.713685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 445/876] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:28.719681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:28.817424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:28.824902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0c058-2ebe-4abb-bdf9-00621f58295f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:28.861435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:28.862049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:28.862331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:28.862604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:28.863235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:28.863538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:28.864918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ced595-e38a-4922-ac6c-14c20f43373f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:28.869976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 432/877] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:28.875535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:28.968619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:28.979973 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b058bd4e-21c3-4eb1-9de8-d9031fe85afd None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:29.014671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:29.015288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:29.015567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:29.015838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:29.016497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:29.017082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:29.018446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31863148-4793-4a34-ae43-318d79528ed1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:29.023357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 446/878] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:29.149277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565f4ba-4541-48dc-8150-774d4c8b3ff7 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:29.149611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565f4ba-4541-48dc-8150-774d4c8b3ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:29.149835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565f4ba-4541-48dc-8150-774d4c8b3ff7 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:29.151467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 433/879] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:29.156997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5e593-d1c4-4981-8823-380c4cf7b36e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:29.157305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5e593-d1c4-4981-8823-380c4cf7b36e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:29.157541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5e593-d1c4-4981-8823-380c4cf7b36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:29.178314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a5a5e593-d1c4-4981-8823-380c4cf7b36e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:29.180469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a5a5e593-d1c4-4981-8823-380c4cf7b36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:29.253583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 447/880] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:29 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:29.278130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565f4ba-4541-48dc-8150-774d4c8b3ff7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:29.389761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565f4ba-4541-48dc-8150-774d4c8b3ff7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:29.425764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a9916-0921-4246-80ad-93fc8b33d983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:29.426378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a9916-0921-4246-80ad-93fc8b33d983 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:29.426936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a9916-0921-4246-80ad-93fc8b33d983 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:29.427207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a9916-0921-4246-80ad-93fc8b33d983 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:29.427864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a9916-0921-4246-80ad-93fc8b33d983 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:29.429853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38a9916-0921-4246-80ad-93fc8b33d983 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:29.503979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 434/881] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:29 2022] POST /identity/v3/services => generated 269 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:29.985739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f1155-6d25-4ac2-9b75-977c45647f59 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:29.986292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f1155-6d25-4ac2-9b75-977c45647f59 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:29.986292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f1155-6d25-4ac2-9b75-977c45647f59 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:29.987859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 448/882] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:29.993406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a4119-53c8-438a-a8c4-95b6e146e26c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:29.993866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a4119-53c8-438a-a8c4-95b6e146e26c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:29.993967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a4119-53c8-438a-a8c4-95b6e146e26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.012777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4c9a4119-53c8-438a-a8c4-95b6e146e26c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:30.013961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4c9a4119-53c8-438a-a8c4-95b6e146e26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:30.027753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:30.089412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 435/883] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:29 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:30.112031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f1155-6d25-4ac2-9b75-977c45647f59 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.198031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:30.217774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f1155-6d25-4ac2-9b75-977c45647f59 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.227970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494704b2-c8d3-4c6c-89b1-3fd12c5a94ab None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.228283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494704b2-c8d3-4c6c-89b1-3fd12c5a94ab None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.228566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494704b2-c8d3-4c6c-89b1-3fd12c5a94ab None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.230168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 436/884] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:30.235284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.235567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.235897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.257639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:30.258843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:30.259325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.260216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.260820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.261099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.261687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:30.263181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:30.273690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:30.274756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:30.275579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 449/885] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:30.282267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.356022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 437/886] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:30.379018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.406487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c338655-d2ad-4d69-8ffe-fac1810ab6d2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.447725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.448733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.449218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.449747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.450594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:30.451101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:30.452683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:30.457998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 450/887] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:30.464173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.473466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:30.480682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:30.505869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab1f11-4d20-4892-857e-66e9abb7fe7b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.557236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.558718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.559187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.559681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.573287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:30.580649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:30.581656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:30.582485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:30.589637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 438/888] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:30.591135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.593373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627985d3-f2a7-4af3-96a1-70e8b6650c91 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.639176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.639788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.640091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.640365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.640992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:30.641294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:30.643205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:30.655387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 451/889] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:30.656773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.729433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdd394-1a1a-4efd-b1e8-91967ee857d0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.752339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:30.776585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.777553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.778071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.778188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570107c9-598d-482e-9025-21d76eeac827 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.778654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.779664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:30.780190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:30.782584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:30.784247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:30.795616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 439/890] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:30.802109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.821663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.822455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.822739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.823019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.823818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] RBAC: Adding query filter params (interface=public, service_id=8795100626284b3f89fc716a4fed38fa, region_id=RegionOne) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:30.824170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:30.825646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:30.832043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 452/891] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/endpoints?service_id=8795100626284b3f89fc716a4fed38fa&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:30.838135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:30.918474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a796f46f-451f-4653-a3c4-2d17932abd8b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.954412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:30.955410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:30.955872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:30.956355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:30.967509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:30.967658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18a8526-35c2-47c9-836d-6bffd3fa655c None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:30.969352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:30.985769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:30.986783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:30.987692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:30.987692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:30.987692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:30.994804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 440/892] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:30.996539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.007042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.007652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.008076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.008411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.009075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.010881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.024103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 453/893] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:04:31.056479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:31.115154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516cee59-9570-4de4-bf8d-655b1438a6c5 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:31.153159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.154123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.154580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.155028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.155855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:31.156386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.157976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1b2cf-aff5-4997-a75e-0a5bde325ac1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.172829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 441/894] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:31.179029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.182546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:31.227671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae75a9e-b8fa-4c7e-a653-3ab164d5f801 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.227963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae75a9e-b8fa-4c7e-a653-3ab164d5f801 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.228256 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae75a9e-b8fa-4c7e-a653-3ab164d5f801 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.229856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 442/895] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:31.235002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.235295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.238407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.256637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:31.258062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:31.292239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50680082-4517-42fc-a7a7-746794ac2a5a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:31.332417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.333180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.333524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.333906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.342358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.343484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 443/896] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:31.352095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:31.356384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:31.357469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:31.358606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:31.360881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 454/897] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:31.366422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.375810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.411565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:31.478665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:31.479730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830c903-b45e-4ed2-9300-ec5cfa40ddff None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:31.524178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f665196-7b7d-4618-bc55-dc9615621367 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:31.527782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ff197-ba79-4c57-b9aa-bddacdfe5a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.528624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ff197-ba79-4c57-b9aa-bddacdfe5a61 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.529043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ff197-ba79-4c57-b9aa-bddacdfe5a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.529455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ff197-ba79-4c57-b9aa-bddacdfe5a61 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.530246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ff197-ba79-4c57-b9aa-bddacdfe5a61 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.532305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ff197-ba79-4c57-b9aa-bddacdfe5a61 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.561420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.562024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.562302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.562574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.563294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:31.563598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.565051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.583935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 455/898] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 10 11:04:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:31.589235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.620619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 444/899] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:31 2022] POST /identity/v3/services => generated 241 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:31.623578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c288-93b7-45ea-9dd4-35ad4e65318f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.623970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c288-93b7-45ea-9dd4-35ad4e65318f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.624330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c288-93b7-45ea-9dd4-35ad4e65318f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.626281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 445/900] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:31.636057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.636057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.636057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.658804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:31.660984 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:31.715930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074255b-020b-483c-b0ca-d6e682cc81f3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:31.739661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:31.764311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.764795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 446/901] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:31.768678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.768961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.769234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.787933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.822506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:31.852907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.854299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.915116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e1c12e-39e8-483c-8c4e-be17a625fa8c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:31.953802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:31.954438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:31.954719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:31.954990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:31.955575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:31.956965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:31.966494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:31.967487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:31.968405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 447/902] 10.222.0.13 () {56 vars in 1132 bytes} [Wed Aug 10 11:04:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:31.973856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:31.987366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 456/903] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:31 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 11:04:31.993595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.012044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:32.079149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:32.092629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb7c50f-bd2b-489e-bd74-5aa636b03b6b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.115009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161d5e3-10e2-4e97-8bc0-6588320e887e None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.132050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.132791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.133080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.133346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.134032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:32.134401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.135770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.141220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 448/904] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:32.146594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.167985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.169430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.169910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.173953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.187659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.189221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:32.205016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:32.206018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:32.206938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:32.206938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:32.206938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:32.206938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 457/905] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:32.211812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.280369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c7151-649d-4fc7-a8ea-d20e8a9eefa7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.284801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:32.325332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4164c68-0d10-423e-9131-e4246903bab4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.327428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.328693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.328975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.329242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.329869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:32.330167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.331514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.342199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 449/906] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:32.343313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.374459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:32.380219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.381152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.381460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.385979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.385979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:32.385979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.385979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.404077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 458/907] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:32.407118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.478855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2f3c3-e569-4c4f-94e1-3746243563fe None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.511859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9a652-7de0-4ba1-9af7-702e73fc5948 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.517165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.517774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.518052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.518323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.519021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] RBAC: Adding query filter params (interface=public, service_id=aa207864726c48d19a73e61440044a15, region_id=RegionOne) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:32.519328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.520775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.531455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 450/908] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/endpoints?service_id=aa207864726c48d19a73e61440044a15&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:32.532685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.546733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:32.556054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.556950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.557445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.557958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.565166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.566838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:32.579231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:32.580437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:32.581348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:32.583515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 459/909] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:32.589952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.639199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:32.661086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5ecfd-bec4-40bc-8100-ad2002f42c2f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.706319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.707094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.707379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.707653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.708300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.710096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.724499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 451/910] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/services => generated 1497 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:04:32.728446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035f3c7-3665-46ff-87d6-6bcb26349c69 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.768866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.770018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.770520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.770996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.772428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:32.772995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.774710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e2eb14-2e35-4098-b0b9-36b5d0566533 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.790378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 460/911] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:32.796927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:32.916797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aec5ee-1fe7-499a-84bc-4b4004d24949 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:32.955187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:32.956201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:32.956683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:32.957156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:32.958068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] RBAC: Adding query filter params (role.id=1f74301298414e66b8138d7093619521, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=832be98ac39748009caa1124bf1b3a4c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:32.958574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:32.960858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d7ea5-abdc-403f-ab1d-b6f723cf1def None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:32.967084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 452/912] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:32 2022] GET /identity/v3/role_assignments?user.id=832be98ac39748009caa1124bf1b3a4c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=1f74301298414e66b8138d7093619521 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:33.191812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9fe46-7e0c-43a2-a599-f2957f2b534a None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.191812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9fe46-7e0c-43a2-a599-f2957f2b534a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.191812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9fe46-7e0c-43a2-a599-f2957f2b534a None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.191812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 461/913] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:33.194434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a50787-c317-47eb-981a-5de12089416f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.194905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a50787-c317-47eb-981a-5de12089416f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.194993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a50787-c317-47eb-981a-5de12089416f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.214482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d6a50787-c317-47eb-981a-5de12089416f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:33.215672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d6a50787-c317-47eb-981a-5de12089416f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:33.217972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:33.288249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 453/914] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:33.308495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9fe46-7e0c-43a2-a599-f2957f2b534a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:33.363271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:33.420306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9fe46-7e0c-43a2-a599-f2957f2b534a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:33.456303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:33.456852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.457130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.457398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.457971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:33.459424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:33.468319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:33.469650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 10 11:04:33.470767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:33.474254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 462/915] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:33.476473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a50787-c317-47eb-981a-5de12089416f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:33.579938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a50787-c317-47eb-981a-5de12089416f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:33.614923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:33.615622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.616551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.616972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.617601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:33.617903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:33.619265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:33.624445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 454/916] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:33.629882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:33.736093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611caa6-c655-4cf1-96b0-e9836a46ac5d None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:33.756506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:33.777258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:33.777874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.778152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.778419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.779040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:33.779339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:33.780716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e0d467-5744-418d-834f-a26a82f425cd None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:33.786091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 463/917] 10.222.0.13 () {56 vars in 1114 bytes} [Wed Aug 10 11:04:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:33.793060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:33.798533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:33.802804 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588dca3-1172-4f77-bd12-4fe91fefd823 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.803753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588dca3-1172-4f77-bd12-4fe91fefd823 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.804182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588dca3-1172-4f77-bd12-4fe91fefd823 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.806206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 464/918] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:33.811036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.815259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.815259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.830586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:33.831825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:33.906389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297563b2-2622-4994-9285-fab8ff649b73 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:33.909302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 465/919] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:33.937987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:33.948572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:33.949376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:33.950227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:33.950663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:33.951921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:33.954148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:34.046152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 455/920] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:34.047966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8891205-ad14-46b5-b1de-79078c71ecf8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:34.058697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:34.099521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:34.100914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.101231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.101574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.102246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:34.105017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:34.115343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:34.116523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:34.118394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 466/921] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:34.123365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:34.197604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:34.242650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79700e0-dec6-4c97-86ff-aa61fdae38ac None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:34.279813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:34.280457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.280733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.281003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.281637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:34.281939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:34.283304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:34.289038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 456/922] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:34.294440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:34.400244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c8eba-3508-4cd9-9b1b-c2a5ae455da5 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:34.435646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:34.436284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.436560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.436830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.437453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:34.437755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:34.439121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:34.444356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 467/923] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:34.450002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:34.556379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9eabc-f8c0-47e0-889f-835d2fec7173 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:34.587155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:34.592153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:34.592834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.593105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.593376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.594079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] RBAC: Adding query filter params (interface=public, service_id=ebb90873146b4828b5bdeb006ada3a73, region_id=RegionOne) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:34.594388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:34.595752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05a7ac-9976-4bbc-b910-d60fff784356 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:34.602105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 457/924] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:34 2022] GET /identity/v3/endpoints?service_id=ebb90873146b4828b5bdeb006ada3a73&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:34.608022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:34.640410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:34.718924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fac1e-fcbd-40f9-b782-15281dee57d1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:34.754957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e020f4c-1dd6-4dbb-8a5d-754932afc10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:34.755960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e020f4c-1dd6-4dbb-8a5d-754932afc10d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.757113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e020f4c-1dd6-4dbb-8a5d-754932afc10d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.757599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e020f4c-1dd6-4dbb-8a5d-754932afc10d None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.758403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e020f4c-1dd6-4dbb-8a5d-754932afc10d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:34.759961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e020f4c-1dd6-4dbb-8a5d-754932afc10d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:34.766857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 468/925] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:34 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:04:34.898384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882ba5-6b77-4108-95b1-4624f358d694 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.898634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882ba5-6b77-4108-95b1-4624f358d694 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.898853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5882ba5-6b77-4108-95b1-4624f358d694 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.900505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 458/926] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:34.905945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c501f-e028-40f0-8180-0aba1913eb95 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:34.906454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c501f-e028-40f0-8180-0aba1913eb95 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:34.906879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c501f-e028-40f0-8180-0aba1913eb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:34.926825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-090c501f-e028-40f0-8180-0aba1913eb95 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:34.928871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-090c501f-e028-40f0-8180-0aba1913eb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:35.002770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 469/927] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:35.026261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5882ba5-6b77-4108-95b1-4624f358d694 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.082584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaa430-31ec-4755-8b2a-8b4eff274990 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.082894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaa430-31ec-4755-8b2a-8b4eff274990 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.083149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaaa430-31ec-4755-8b2a-8b4eff274990 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.085083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 470/928] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:35.090124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.090412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.090693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.109921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:35.111145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:35.134929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:35.145524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5882ba5-6b77-4108-95b1-4624f358d694 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.189930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.190793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.191220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.191623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.192301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.193917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:35.196101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 471/929] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:35.207732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:35.215662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:35.216931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 459/930] 10.222.0.13 () {56 vars in 1132 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:35.218244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.219816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.276499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:35.333157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0c63-2326-49ea-94ff-61a8836e62eb None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.352351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be307ef-cab2-4a8a-ac89-e853dfbdac29 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.377633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.378512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.379085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.379559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.390752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.391358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.391895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.392205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.392844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:35.393142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.394663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:35.399885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:35.401166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 472/931] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:35.405794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:35.406751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:35.407616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:35.408323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.416095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 460/932] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:35.418798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.428222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:35.519346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f129d-db73-4fca-8bd6-ccc0b975e322 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.559665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a7a78c-2de0-4716-911a-59a3b265686c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.568757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.570175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.571183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:35.571775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.572453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.573580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:35.574080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.576413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:35.588091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 473/933] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:35.589662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.602414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.602749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.603504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.604328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.604997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:35.605300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.606657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:35.620176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 461/934] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:35.621046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.699075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:35.722851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68954ef4-db73-41e5-82e5-999244276eb3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.742904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cb9433-6a84-4f90-b688-ba1498d96873 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.761825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.762801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.763280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.763760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.764656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.766219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:35.781122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.781695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.781987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.782243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.794095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.799845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:35.811054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:35.812095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:35.813119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:35.813119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:35.813119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:35.813977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 462/935] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:35.817727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:35.844076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:35.872484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 474/936] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:35.938246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a860f66c-bf14-424e-8c29-56794a227896 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:35.987048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:35.988095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:35.988633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:35.989117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:35.990026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:35.991185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:35.992948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.007187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 463/937] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:36.013460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.122317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e82a6-4b6e-43ff-985d-e43c96d90671 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.158269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.159204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.159653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.160132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.168756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.170371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.177577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-248c6ee2-16f1-4bc5-9525-7a8ce80cbfe5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:36.182922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:36.184152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:36.185323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.187608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 475/938] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:36.193369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.279304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c2994-f436-4919-911f-90eafb2809b9 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.279641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c2994-f436-4919-911f-90eafb2809b9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.279868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c2994-f436-4919-911f-90eafb2809b9 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.281498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 476/939] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:36.286060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:36.286510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.286590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.286824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.306926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269a1716-dbfd-404b-a2d3-378929225a01 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.308818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:36.310533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:36.342277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.342887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.343173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.343445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.344542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:36.344859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.346273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.365464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 464/940] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:36.373032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.402351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 477/941] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:36.426740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.505495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f042d2-b4f4-4f39-90f9-f09054b24887 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.533812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba61f4-bdcb-48d6-9b3d-6095ad574afc None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.552076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.553717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.554063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.554411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.564264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:36.565285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:36.570373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.572239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.574175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.575060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.575440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.575855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.576610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.578168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.581094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING keystone.server.flask.common [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:36.589117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:36.596736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:36.597615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:36.597615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:36.597615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:36.597615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 465/942] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:36.599231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.653164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.sql.core [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 11:04:36.653164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 11:04:36.653164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [parameters: {'id': 'cefb9bdc1a2d4e02a4d1e9600c32b557', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 11:04:36.653164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cefb9bdc1a2d4e02a4d1e9600c32b557.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cefb9bdc1a2d4e02a4d1e9600c32b557. Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:04:36.674660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result.read() Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 10 11:04:36.675572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application yield resource Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise self.value Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application session.commit() Aug 10 11:04:36.677053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.session.flush() Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application compat.raise_( Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 11:04:36.678083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 11:04:36.681572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result.read() Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:04:36.683811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [parameters: {'id': 'cefb9bdc1a2d4e02a4d1e9600c32b557', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.686095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:36.688172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cefb9bdc1a2d4e02a4d1e9600c32b557. Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 478/943] 10.222.0.13 () {60 vars in 1140 bytes} [Wed Aug 10 11:04:36 2022] POST /identity/v3/projects => generated 262 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:36.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.724578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f33fe-a1a0-40d3-936a-675517a64c43 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.768583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.769675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.769966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.770236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.770890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:36.771185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.774277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.790873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 466/944] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:36.794966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.819023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131a94c-4ce2-4e56-88cb-2e580990256f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.855390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:36.862319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.863366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.863488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.863824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.872222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.873875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-02fac1b5-15a8-45cb-a35b-308754a09abb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:36.886055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:36.887083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:36.888089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:36.889789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 479/945] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:36.891164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2791004-5eb0-4422-8fe8-275c2c22ddbc None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.891586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2791004-5eb0-4422-8fe8-275c2c22ddbc None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.891988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2791004-5eb0-4422-8fe8-275c2c22ddbc None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.896168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 480/946] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:36.897118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2791004-5eb0-4422-8fe8-275c2c22ddbc None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:36.921967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6db8-2b2a-450a-8691-bd996f8f0533 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:36.945631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:36.966452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:36.967085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.967364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.967631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:36.975402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:36.976885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a3ef5716-80e0-428b-a1b4-234e7db254f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:36.988586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:36.990937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 467/947] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:36.992086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:36.992637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:36.992637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.008524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2791004-5eb0-4422-8fe8-275c2c22ddbc None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.012041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:37.013230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:37.043333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.043938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.044251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.044519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.045170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:37.045469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.046912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.064656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 481/948] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:36 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:37.065482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:37.107297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 468/949] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:37.130596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:37.184803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427028-8c96-4c2a-ab44-cb2343888a28 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.187310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:37.198321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:37.226449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.227319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.227700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.228576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.229681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:37.230094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.231848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.248349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 482/950] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:37.253974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:37.261938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3114a5-525f-48cd-af50-ef5989228a8b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.305046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.307083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.307503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.307920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.309102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.312730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:37.325219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:37.326258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:37.327315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 469/951] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:37.332647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:37.380102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a89e6-022f-4388-9d00-8455855738f9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.422919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.424409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.424917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.425404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/832be98ac39748009caa1124bf1b3a4c/roles/1f74301298414e66b8138d7093619521` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.429460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=832be98ac39748009caa1124bf1b3a4c, role_id=1f74301298414e66b8138d7093619521)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.431181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.456361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fbd25f-a44d-4023-9383-1039261458df None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.460064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:37.495817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.496602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.496981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.497366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.498084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:37.498503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.500033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.509695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 470/952] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:37.510721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:37.556907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 483/953] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:37 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/832be98ac39748009caa1124bf1b3a4c/roles/1f74301298414e66b8138d7093619521 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:04:37.641612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cff32e-b357-413b-8924-1940ec6e3492 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.685661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.686685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.686963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.687279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.687990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:37.688313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.689688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b8140-5d54-4f3c-b5b2-e0dcc88aac84 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.696711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 471/954] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:37.702469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:37.736447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:37.814552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259df985-0f7f-4d2c-a840-67a095d0fbdb None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:37.850547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa1fdc-1d7e-4abf-a6e5-107e1fce179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:37.851157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa1fdc-1d7e-4abf-a6e5-107e1fce179f None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:37.851434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa1fdc-1d7e-4abf-a6e5-107e1fce179f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:37.851703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa1fdc-1d7e-4abf-a6e5-107e1fce179f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:37.852410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa1fdc-1d7e-4abf-a6e5-107e1fce179f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:37.853768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa1fdc-1d7e-4abf-a6e5-107e1fce179f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:37.935414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 484/955] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:38.049351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36d405-ef3e-4ab8-abb6-b0773202ceea None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:38.049546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36d405-ef3e-4ab8-abb6-b0773202ceea None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:38.049803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36d405-ef3e-4ab8-abb6-b0773202ceea None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:38.051331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 472/956] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:38.057963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:38.058277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:38.058518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:38.078072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:38.079628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:38.157868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 485/957] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:38.181407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36d405-ef3e-4ab8-abb6-b0773202ceea None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:38.214524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:38.289058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36d405-ef3e-4ab8-abb6-b0773202ceea None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:38.326773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:38.327395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:38.327677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:38.327948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:38.328566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:38.329937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:38.339616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:38.340790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:38.341612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 473/958] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:38.347590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:38.357041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:38.465499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b53b2-79bc-46ba-b8f0-cbac67bdfcde None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:38.499951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:38.500589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:38.500861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:38.501128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:38.501774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:38.502044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:38.503393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f9fd1-7dc5-4428-8973-556d4702e10d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:38.508515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 486/959] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:38.514248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:38.612791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:38.618571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5513698-05dc-40fd-896b-7b2438934f44 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:38.654297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:38.654904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:38.655183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:38.655454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:38.656108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] RBAC: Adding query filter params (type=compute) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:38.656414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:38.657770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1599f-22f6-4bcb-bf0b-d286c28d33ea None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:38.662895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 474/960] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:38 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:39.290941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280ce8c-9175-496c-bef3-fa0ce81b855e None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.293144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280ce8c-9175-496c-bef3-fa0ce81b855e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.293144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280ce8c-9175-496c-bef3-fa0ce81b855e None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.293144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 487/961] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:39.298727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.299181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.299658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.319406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:39.320885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:39.400111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 475/962] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:39.419023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280ce8c-9175-496c-bef3-fa0ce81b855e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:39.496915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:39.528522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280ce8c-9175-496c-bef3-fa0ce81b855e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:39.564043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:39.564571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.564840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.565105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.565747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:39.567159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:39.618627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.sql.core [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 11:04:39.618627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 11:04:39.618627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [parameters: {'user_id': '43205c8af6c34302ad9990d0fce48c4f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 11:04:39.618627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-3fbc39af-b0e6-4317-9249-def9cfe8f5a8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result.read() Aug 10 11:04:39.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:04:39.626298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application yield resource Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application next(self.gen) Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.session.rollback() Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.force_reraise() Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise self.value Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application session.commit() Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 10 11:04:39.627383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._prepare_impl() Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.session.flush() Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._flush(objects) Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application compat.raise_( Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application flush_context.execute() Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rec.execute(self) Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 10 11:04:39.628556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 10 11:04:39.629652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result = self._query(query) Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application conn.query(q) Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application result.read() Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 10 11:04:39.630684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [parameters: {'user_id': '43205c8af6c34302ad9990d0fce48c4f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:39.631872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 10 11:04:39.632911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:39.633927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 488/963] 10.222.0.13 () {60 vars in 1131 bytes} [Wed Aug 10 11:04:39 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:39.634336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:39.728818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a66422-a551-4f5e-a786-99c19090b7a5 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.729886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a66422-a551-4f5e-a786-99c19090b7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.730164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a66422-a551-4f5e-a786-99c19090b7a5 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.730942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:39.735787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 489/964] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:39.741760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff36c19-dfe8-4e76-9230-4935c310df9d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:39.742744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.743187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.747066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.767019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:39.768894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:39.785336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:39.785975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.786255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.786532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.795420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:39.797238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:39.810807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:39.811851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 10 11:04:39.812959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:39.815392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 476/965] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:39 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:39.824129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:39.852099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 490/966] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:39.876875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:39.945241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:39.952729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a49b33-a8b4-4430-832d-e992e582fb89 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:39.993454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:39.994451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:39.994927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:39.995389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:39.996392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ce8cc-a8d3-41b2-a084-80f564036610 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:39.996610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:39.997104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:39.998775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.033798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a60e9c-0354-4abf-aaf3-66598bb84681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.034434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a60e9c-0354-4abf-aaf3-66598bb84681 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.034710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a60e9c-0354-4abf-aaf3-66598bb84681 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.034982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a60e9c-0354-4abf-aaf3-66598bb84681 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.041762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 477/967] 10.222.0.13 () {56 vars in 1101 bytes} [Wed Aug 10 11:04:39 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:40.055811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a60e9c-0354-4abf-aaf3-66598bb84681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-f7a60e9c-0354-4abf-aaf3-66598bb84681 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:40.061600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:40.062601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:40.063457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:40.066543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 491/968] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 10 11:04:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:40.073762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.087198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.112882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dce36-52dd-44fa-bc0a-406348606700 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.113502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dce36-52dd-44fa-bc0a-406348606700 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.113969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dce36-52dd-44fa-bc0a-406348606700 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.116511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 492/969] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:40.121368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.121943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.122465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.148501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:40.150356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:40.207650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3d33f-4d39-462e-a661-7c8e691b6753 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.241352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 493/970] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:40.255804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.257287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.257287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.257651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.258192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:40.258493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.259849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.264726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.269211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 478/971] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:40.276851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.325113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.364097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.380053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9533aac2-38d4-4050-b9ac-649363fc524d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.398263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc2de01-d21f-4886-8a2c-9bafe617f1de None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.418269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.418901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.419182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.419453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.420054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.421557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:40.431540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:40.432562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:40.434558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 494/972] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:40.438550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.439754 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.440544 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.440883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.441328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.454865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.456397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:40.470907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:40.472201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:40.473280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:40.473280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:40.473280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:40.476146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 479/973] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:40.479557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.545498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55beaf0a-ed87-4b14-948f-03ce4eaafa39 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.564355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.582149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.582735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.583000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.583213 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.583829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:40.584679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.586051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.596208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a643138-902d-4318-82cc-b28b75ac79af None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.596372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 495/974] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:40.597045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.620212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.633742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.638875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.638875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.638875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.638875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:40.638875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.638875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.649841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 480/975] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:40.655369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.711759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ea232-9a7c-453a-8872-a7350605df7a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.749509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.750148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.750427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.750961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.751320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:40.751622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.753017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.759047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 496/976] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:40.764193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8982c-043f-416d-bf1b-7497de121719 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.764527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.799196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.799816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.800120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.800391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.808403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.809335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.809886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-24904176-ee56-4dd5-9930-74dfa248ea08 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:40.821688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:40.828179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 481/977] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05647d00-bcf4-4ec8-b89f-39a2389e18a2 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.829330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05647d00-bcf4-4ec8-b89f-39a2389e18a2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.830133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05647d00-bcf4-4ec8-b89f-39a2389e18a2 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.830133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 482/978] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:40.830133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05647d00-bcf4-4ec8-b89f-39a2389e18a2 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:40.872031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536b337-bea2-4386-b223-2f008ce315e6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.873086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:40.905871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.906476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.906756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.907026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.907744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] RBAC: Adding query filter params (interface=admin, service_id=ebb90873146b4828b5bdeb006ada3a73, region_id=RegionOne) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:40.908088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.909440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635a407b-a669-4b7f-850c-efa80d701e07 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.919478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 497/979] 10.222.0.13 () {56 vars in 1249 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/endpoints?service_id=ebb90873146b4828b5bdeb006ada3a73&interface=admin®ion_id=RegionOne => generated 193 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:40.919478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.919478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.919478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.931757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05647d00-bcf4-4ec8-b89f-39a2389e18a2 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:40.934768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:40.935956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:40.965659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:40.966256 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:40.966529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:40.966795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:40.967506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:40.967808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:40.969249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:40.986260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 483/980] 10.222.0.13 () {56 vars in 1149 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:40.987043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.001913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 498/981] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:41.002739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.045666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:41.088981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:41.092216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57b6065-6eca-4a6e-ab94-1d62eca12f49 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:41.107873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b976217-df80-4bee-ae5d-0b6646d3bf88 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:41.129227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:41.129846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:41.130130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:41.130403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:41.131057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:41.132466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:41.138439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 484/982] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:40 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:04:41.139186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.159342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:41.161538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:41.162779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:41.162779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:41.246860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:41.248369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:41.278830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3db56-7879-4538-8077-5377d8146605 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:41.291160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:41.323062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:41.323722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:41.324032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:41.324302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:41.324950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:41.326314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:41.357776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 499/983] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:41.363422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.404267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 485/984] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:41 2022] POST /identity/v3/services => generated 242 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:41.475199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74c92b-9806-4368-99b7-7dacfd636ff0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:41.521024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:41.521716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:41.521991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:41.522258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:41.524561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:41.535079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:41.536578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:41.551562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:41.553709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:41.555118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:41.555118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:41.555118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:41.555118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 500/985] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:41.559374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.592416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:41.669464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d85409-e451-47ec-8048-f1849a8ed31c None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:41.705950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:41.706556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:41.706834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:41.707100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:41.707737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:41.708074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:41.709530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:41.721290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 486/986] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:41.727548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.830592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff696d-e119-4999-873c-df46dcf75b15 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:41.864675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:41.865359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:41.866177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:41.866449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:41.873319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:41.874764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:41.886651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:41.887585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:41.888920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 501/987] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:41.893455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:41.984052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:41.999935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd603f47-fc51-4104-af9d-7ab5d1851036 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:42.034072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:42.034674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:42.034953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:42.035222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:42.035901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:42.036235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:42.038426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2c8ce-11a9-49a8-a1f0-a580b7a0ed55 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:42.049937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 487/988] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:42.056313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:42.105142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:42.158192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ecea9-b5d6-40f8-af2f-b26a74fc96ed None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:42.192352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:42.193003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:42.193278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:42.193870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:42.194634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] RBAC: Adding query filter params (role.id=1f74301298414e66b8138d7093619521, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=832be98ac39748009caa1124bf1b3a4c) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:42.194936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:42.196441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c90eb1-5610-479b-9686-036c35d418c3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:42.208034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 502/989] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:42 2022] GET /identity/v3/role_assignments?user.id=832be98ac39748009caa1124bf1b3a4c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=1f74301298414e66b8138d7093619521 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:43.170885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286e67-1679-4ca5-b111-2ea078ab947e None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.171411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286e67-1679-4ca5-b111-2ea078ab947e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.171411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3286e67-1679-4ca5-b111-2ea078ab947e None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.173059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 488/990] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:43.183148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.183148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.183148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.199751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:43.201007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:43.280344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 503/991] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:43.300043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286e67-1679-4ca5-b111-2ea078ab947e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:43.424059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3286e67-1679-4ca5-b111-2ea078ab947e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:43.460014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:43.460767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.461117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.461479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.462130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:43.463553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-0efbf3af-afec-4872-9d50-07884227e142 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:43.472778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:43.473759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 10 11:04:43.475218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:43.479075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 489/992] 10.222.0.13 () {56 vars in 1126 bytes} [Wed Aug 10 11:04:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:43.481262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:43.506888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f599e-365b-457e-96a7-3c420741f159 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.507197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f599e-365b-457e-96a7-3c420741f159 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.507431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f599e-365b-457e-96a7-3c420741f159 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.508894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 490/993] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:43.513836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.514302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.514412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.531277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:43.535175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:43.536636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:43.541232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:43.608809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be883dd-7c23-4378-8b6f-014f24f6ead6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:43.621016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 491/994] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:43.643550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:43.644191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.644466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.644727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.644992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:43.645345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:43.645640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:43.647627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:43.657155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 504/995] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:43.658153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:43.750773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb60b9-5886-4639-921b-24d33009ec7e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:43.765133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34df9f-f5a5-48ae-86e5-8ea5813d4e45 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:43.768091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:43.789817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:43.790458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.795363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.795363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.795363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:43.795363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:43.801981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:43.803499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:43.804382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:43.805152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:43.805152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.805313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 492/996] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:43.805383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.805383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.805383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:43.806656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:43.807238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:43.809523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:43.812561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:43.814097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 505/997] 10.222.0.13 () {56 vars in 1122 bytes} [Wed Aug 10 11:04:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:43.818854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:43.916571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b96a191-e071-4025-bd1a-200fe8796e2a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:43.923806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90036691-3631-43ac-967f-59477bbf7ac0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:43.954796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:43.955413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.955694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.955965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.956636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:43.956977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:43.960262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:43.962337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:43.963110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:43.963529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:43.963949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:43.964635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:43.965982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:43.971857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 493/998] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:43.972884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:44.038203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 506/999] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:04:44.039807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:44.087138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ef415-e4f9-4691-90cc-71caf856e4a2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:44.132056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:44.132770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.133054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.133329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.133970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:44.134274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:44.135655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287cfb9c-28d6-43f0-90dd-023868963fb9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:44.141497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 494/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:43 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:44.148036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:44.179373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:44.257687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a35d71-77f0-4e63-a1db-1122a5906a02 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:44.277938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e529d-2c0b-4b73-a027-7e72c61cc7f4 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.278919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e529d-2c0b-4b73-a027-7e72c61cc7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.279888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1e529d-2c0b-4b73-a027-7e72c61cc7f4 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.282255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 495/1001] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:44.287439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.287952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.288730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.293951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:44.294561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.294848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.295125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.295823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] RBAC: Adding query filter params (interface=public, service_id=0465e82aeafe436ba2cf6f7221970e91, region_id=RegionOne) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:44.296260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:44.297620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:44.303562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 507/1002] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:44 2022] GET /identity/v3/endpoints?service_id=0465e82aeafe436ba2cf6f7221970e91&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:44.309175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:44.317421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:44.319667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:44.408412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 496/1003] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:44 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:44.412348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:44.419881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:44.421155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59d06d-bb7e-40fa-9a79-e4f9983048a2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:44.459409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:44.460523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.460802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.461070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.461697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:44.463130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:44.470728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 508/1004] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:44 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:04:44.482098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:44.534323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b138ae7-b000-455c-9a09-e650ddf4ebbb None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:44.572614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:44.573269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.573550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.573823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] PATH_INFO: `/v3/services/image` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.574419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:44.575799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:44.585445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:44.586523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:44.592921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 497/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Wed Aug 10 11:04:44 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:44.593713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:44.700948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c7ad6-33d7-4c9f-8988-60f2d38ebc08 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:44.735591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:44.736234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.736513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.736785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.737467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:44.737766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:44.739151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af82ed-3ae6-413c-a0a1-18dc8cfddea2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:44.749178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 509/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:44.750403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:44.860766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297049bc-9677-40f8-b0f9-433629784cae None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:44.880512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:44.894356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:44.894955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:44.905526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:44.905526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:44.905526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] RBAC: Adding query filter params (type=image) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:44.905526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:44.905526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f028ff73-a0dc-48e5-9c9b-9b385c38512f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:44.905526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 498/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:44 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:46.098040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b509d-4669-4500-a141-f9e971463dea None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.098567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b509d-4669-4500-a141-f9e971463dea None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.098567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b509d-4669-4500-a141-f9e971463dea None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.100854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 510/1008] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:46.106086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.106503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.106670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.124710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:46.125885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:46.196553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 499/1009] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:46.219957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b509d-4669-4500-a141-f9e971463dea None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:46.241883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:46.323190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b509d-4669-4500-a141-f9e971463dea None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:46.357099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2388f-4b7e-4ee5-a936-b5f9da602b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:46.357651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2388f-4b7e-4ee5-a936-b5f9da602b7b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.357929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2388f-4b7e-4ee5-a936-b5f9da602b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.358197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2388f-4b7e-4ee5-a936-b5f9da602b7b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.359314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2388f-4b7e-4ee5-a936-b5f9da602b7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:46.360757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b2388f-4b7e-4ee5-a936-b5f9da602b7b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:46.362470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 511/1010] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:46.369630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:46.472865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc00d49-8321-4511-9cc0-d19a94b6f0a5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:46.487365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:46.508550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:46.509513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.509991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.510460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.511310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:46.512945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:46.542393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33667a5-9963-4704-aab8-6b447dea171c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.542678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33667a5-9963-4704-aab8-6b447dea171c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.542950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33667a5-9963-4704-aab8-6b447dea171c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.544326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 512/1011] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:46.549086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.549365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.549639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.568364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:46.570188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:46.635928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 513/1012] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:46.650189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 500/1013] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:46 2022] POST /identity/v3/projects => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:46.658586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:46.679143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:46.767934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f8239-65ad-449a-a711-78f5011afeb8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:46.804951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:46.805840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.806160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.806441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.807020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:46.808425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2a9c34f6-fa63-432a-8c93-994c6dc4717a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:46.817294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:46.819384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:46.820457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 514/1014] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:46.827484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:46.921336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381f7b8-ebdc-4f4b-9e6f-f740e26a0625 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.921527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381f7b8-ebdc-4f4b-9e6f-f740e26a0625 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.921806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5381f7b8-ebdc-4f4b-9e6f-f740e26a0625 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.923709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 515/1015] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:46.929235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.929520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.929811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.936364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2a3d6-e053-4d4d-86a0-3de56193eeed None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:46.949708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:46.950907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:46.972708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:46.973560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:46.973839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:46.974106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:46.975249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:46.975589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:46.976998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:46.986331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 501/1016] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:46.988092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:47.020422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:47.027833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 516/1017] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:46 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:47.051690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:47.066858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:47.095349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2b779-1d64-45cd-b5fe-0c767b7e2e5e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:47.131819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:47.132948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:47.132948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:47.133160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:47.133704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:47.134006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:47.135359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:47.145513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 502/1018] 10.222.0.13 () {56 vars in 1112 bytes} [Wed Aug 10 11:04:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:47.146495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:47.164563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1674954e-1db7-4a1d-af6a-a3d69da776e6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:47.203467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2c788-70ae-4fe6-9045-adb48234b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:47.204008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2c788-70ae-4fe6-9045-adb48234b7bf None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:47.204298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2c788-70ae-4fe6-9045-adb48234b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:47.204562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2c788-70ae-4fe6-9045-adb48234b7bf None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:47.205219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2c788-70ae-4fe6-9045-adb48234b7bf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:47.206547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2c788-70ae-4fe6-9045-adb48234b7bf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:47.246454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:47.253131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e134cbba-438f-43b4-b1d1-56eb68136230 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:47.288119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60c6ed-45d6-40ba-99e6-5d72f41bd434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:47.288721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c6ed-45d6-40ba-99e6-5d72f41bd434 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:47.288997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c6ed-45d6-40ba-99e6-5d72f41bd434 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:47.289259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60c6ed-45d6-40ba-99e6-5d72f41bd434 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:47.290152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60c6ed-45d6-40ba-99e6-5d72f41bd434 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:47.292336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60c6ed-45d6-40ba-99e6-5d72f41bd434 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:47.333587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 517/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:04:47 2022] POST /identity/v3/services => generated 242 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:04:47.424034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 503/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:47 2022] POST /identity/v3/endpoints => generated 334 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:48.730831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd6c77-f9ee-4624-9ff2-bdd2ec806171 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:48.731341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd6c77-f9ee-4624-9ff2-bdd2ec806171 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:48.731341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd6c77-f9ee-4624-9ff2-bdd2ec806171 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:48.732793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 518/1021] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:48.737700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:48.738078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:48.738252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:48.757064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:48.758236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:48.832520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 504/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:48.856383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd6c77-f9ee-4624-9ff2-bdd2ec806171 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:48.926355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:48.957841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbd6c77-f9ee-4624-9ff2-bdd2ec806171 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:48.993161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf953246-768d-4e9d-ab25-b678bb3d761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:48.993717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf953246-768d-4e9d-ab25-b678bb3d761e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:48.993992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf953246-768d-4e9d-ab25-b678bb3d761e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:48.994261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf953246-768d-4e9d-ab25-b678bb3d761e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:48.995515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf953246-768d-4e9d-ab25-b678bb3d761e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:48.996951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf953246-768d-4e9d-ab25-b678bb3d761e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:48.998670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 519/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:49.004497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.082769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:49.105611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2399d35-0995-4e9f-958e-c0766bfd71cb None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:49.144114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b960-6e6d-4c03-aaf3-7f1e12ee7418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:49.144785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b960-6e6d-4c03-aaf3-7f1e12ee7418 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.145070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b960-6e6d-4c03-aaf3-7f1e12ee7418 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.145336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b960-6e6d-4c03-aaf3-7f1e12ee7418 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.145989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b960-6e6d-4c03-aaf3-7f1e12ee7418 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:49.147587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b960-6e6d-4c03-aaf3-7f1e12ee7418 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:49.228028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 505/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:04:49 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:04:49.385892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ef34-3a16-4460-a422-61d8a0b98ac7 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.386171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ef34-3a16-4460-a422-61d8a0b98ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.386442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be4ef34-3a16-4460-a422-61d8a0b98ac7 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.387915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 520/1025] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:49.393004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269d216-b65d-44f8-bbea-2867cebcb633 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.393284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269d216-b65d-44f8-bbea-2867cebcb633 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.393558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269d216-b65d-44f8-bbea-2867cebcb633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.413002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9269d216-b65d-44f8-bbea-2867cebcb633 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:49.414262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9269d216-b65d-44f8-bbea-2867cebcb633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:49.486448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 506/1026] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:49.510881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4ef34-3a16-4460-a422-61d8a0b98ac7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.546193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba5d9c-4ca8-4d61-b8a3-e1a4ec071f83 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.546490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba5d9c-4ca8-4d61-b8a3-e1a4ec071f83 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.546739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba5d9c-4ca8-4d61-b8a3-e1a4ec071f83 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.548100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 507/1027] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:49.554185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.554459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.554715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.574281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:49.575445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:49.619860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be4ef34-3a16-4460-a422-61d8a0b98ac7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:49.637686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:49.650342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 508/1028] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:49 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:49.653937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:49.654647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.654859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.655129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] PATH_INFO: `/v3/services/image` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.655709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:49.657108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:49.666643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:49.667648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:49.668555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 521/1029] 10.222.0.13 () {56 vars in 1105 bytes} [Wed Aug 10 11:04:49 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:49.675621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.676284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.679949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:49.780547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e48f-a5d6-497b-ab3b-1496d3a446ce None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:49.782834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdccbb9-9208-434f-97d8-adacfb9130c5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:49.817797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:49.818772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.819196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.819317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:49.819829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.819945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.820150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.820421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.821047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] RBAC: Adding query filter params (name=image) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:49.821336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:49.822727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:49.827487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 509/1030] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:49.832989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.837696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:49.842680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:49.843630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:49.844539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 522/1031] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 10 11:04:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:49.850232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.897262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:49.915113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:49.948936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2355a6-d148-4f81-8337-aeab1169ed3b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:49.953930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360fa6b1-c251-4eee-9ff2-ad5d757648df None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:49.983825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:49.984469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.984748 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.985022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.985648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] RBAC: Adding query filter params (type=image) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:49.985947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:49.987313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:49.992320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 510/1032] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:49.994264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:49.995076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:49.995522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:49.995963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:49.996903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:49.997386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:49.997949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:49.999123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.009989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 523/1033] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:04:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:50.015028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:50.111628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a1c51e-132a-4fdf-8352-1ae72acd6138 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.116937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7176e87-9b68-4e62-aa17-3ba175c4eb39 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.132535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:50.143000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:50.147760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.148335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.148615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.148885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.149581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] RBAC: Adding query filter params (interface=public, service_id=a135b986a94f45688a440246142cd524, region_id=RegionOne) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:50.149888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.151233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.152266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.153075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.153354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.153613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.156296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 511/1034] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:04:49 2022] GET /identity/v3/endpoints?service_id=a135b986a94f45688a440246142cd524&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:50.161966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:50.165547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.166974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:50.182000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:50.182998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:50.183909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:50.183909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:50.183909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:50.187897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 524/1035] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:50.188741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:50.275266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44499263-b40d-43f4-a666-96dffa179d3b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.300755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd70ed6-0b89-4757-94b0-dbc274957c6d None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.309439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.310044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.310316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.310583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.311172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.312560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.319174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 512/1036] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:04:50 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:04:50.345417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.346193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.346543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.346882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.347659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:50.348051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.349568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.367442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 525/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:50.370408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:50.428281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:50.485738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f897a-6089-4367-9d9b-d388af0b1621 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.520523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.521150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.522031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.522222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.529913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.531339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:50.543738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:50.544872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:50.545831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 513/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:50.550847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:50.591752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:50.652505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925e4c1-a2c1-451a-a94c-ab5d3c21f4e1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.687690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.688273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.688553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.688920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.689644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:50.689949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.691372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:50.714523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 526/1039] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:50.715788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:50.819805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff87fa-c6ba-4d61-b0f0-79a15eb88c5f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:50.828966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:50.854068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:50.854677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:50.854956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:50.855228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:50.923238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:50.924732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.029379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 514/1040] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:50 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:51.030448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.124161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:51.130462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82feb3bd-d803-4de8-b86e-0e4a2f682c6b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.164397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.165028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.165306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.165578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.175765 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.178573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-7eca39ba-f42e-4429-8d8f-a97445ed5ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:51.193116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:51.195027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:51.195897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:51.195897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:51.195897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:51.195897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 527/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:51.201781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.212139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:51.304193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f938-f05c-473f-a730-53d0a6bc50ba None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.323039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d3e0-61e3-4cf5-90a9-3f61cb012022 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.323316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d3e0-61e3-4cf5-90a9-3f61cb012022 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.323579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6d3e0-61e3-4cf5-90a9-3f61cb012022 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.325114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 528/1042] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:51.330302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.330603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.330881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.342888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.343899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.344204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.344477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.345114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:51.345412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.346757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.348648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:51.350120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:51.358742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 515/1043] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:51.363980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.435177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 529/1044] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:51 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:51.442266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:51.458636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.480013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1df42-3453-4c37-9a14-5dc80e9fcdbe None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.515568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.516229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.516509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.516771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.523637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.525087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:51.536075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:51.537518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:51.538437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:51.542665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 516/1045] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:51.543282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:51.543400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.566495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2dfa1-6324-45c8-9bee-11912ccb8259 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.603043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.603693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.604039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.604285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.614783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:51.620555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:51.621470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:51.622349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 530/1046] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:51.629193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.657615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc32b3-a0f2-4353-8afc-9f07cc0fcdb1 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.690561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:51.699166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.699769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.700183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.700451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.701194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:51.701501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.702908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.715488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 517/1047] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:51.721376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.745194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bafa4-e7b4-4234-85e0-ac9c430e7635 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.781719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.782418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.782597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.782869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.783710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:51.783832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.785194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.791763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:51.795597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 531/1048] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:51.801202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:51.831361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc318ea-6cd6-410d-8200-36c0d85025b9 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.871860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.872701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.872982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.873251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.873980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] RBAC: Adding query filter params (role.id=1f74301298414e66b8138d7093619521, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=462638e8386c49668290954c0211728c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:51.874279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.875693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.882364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 518/1049] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/role_assignments?user.id=462638e8386c49668290954c0211728c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=1f74301298414e66b8138d7093619521 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:51.906350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a750ad-9949-43b3-89fe-1195fa56a0b6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:51.944910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:51.945615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:51.945894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:51.946162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] PATH_INFO: `/v3/users/2609e262ca9747af8655d92cd5d12aa7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:51.961378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] RBAC: Authorizing `identity:get_user(user_id=2609e262ca9747af8655d92cd5d12aa7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:51.962859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:51.964786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 532/1050] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/users/2609e262ca9747af8655d92cd5d12aa7 => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:51.970703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:52.030763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:52.082502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35cc564-2488-46a1-ae52-8aa508901638 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:52.117745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03390249-f0f1-440e-aa82-e30739e5707d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:52.118360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390249-f0f1-440e-aa82-e30739e5707d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.118640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390249-f0f1-440e-aa82-e30739e5707d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.118911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03390249-f0f1-440e-aa82-e30739e5707d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.189720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390249-f0f1-440e-aa82-e30739e5707d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:52.191189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03390249-f0f1-440e-aa82-e30739e5707d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:52.244038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:52.297568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 519/1051] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:51 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 11:04:52.299197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:52.359961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:52.374475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbf8b1-b7b4-4399-8225-292391d00789 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.374475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbf8b1-b7b4-4399-8225-292391d00789 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.374693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efbf8b1-b7b4-4399-8225-292391d00789 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.375826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 520/1052] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:52.380803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.381093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.381370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.399180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:52.400415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:52.410091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f710354e-e753-41a4-b38f-0195990afaf8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:52.449664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:52.450419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.450699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.450961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.452137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] RBAC: Authorizing `identity:get_project(project_id=d1fb6322092b48edb5bd1a86706972f4)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:52.453610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:52.455280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 533/1053] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:04:52 2022] GET /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:52.461495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:52.471351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 521/1054] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:52.495128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:52.566534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366a1a5-ce12-4314-aa02-c12414fe3ed6 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:52.577980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:52.594378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:52.596306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484fbf1-6700-4316-a04e-763bb3c785b4 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:52.603863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:52.604497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.604780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.605049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.605767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=d1fb6322092b48edb5bd1a86706972f4, user.id=2609e262ca9747af8655d92cd5d12aa7) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:52.606071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:52.607493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:52.620318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 534/1055] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:52 2022] GET /identity/v3/role_assignments?user.id=2609e262ca9747af8655d92cd5d12aa7&scope.project.id=d1fb6322092b48edb5bd1a86706972f4&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:52.642132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:52.642788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.643073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.643335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] PATH_INFO: `/v3/services/image` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.643927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:52.645368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:52.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:52.656897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:52.658365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 522/1056] 10.222.0.13 () {56 vars in 1105 bytes} [Wed Aug 10 11:04:52 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:52.663710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:52.778327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8ed7a-1837-4a7d-8b36-7368a98b5b84 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:52.811088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:52.811689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.811964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.812268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.812881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:52.813174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:52.814523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:52.820193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 535/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:52.825773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:52.925166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f30af97-5a76-4d11-b1bc-261983410486 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:52.959757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:52.960290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:52.960566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:52.960834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:52.961455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] RBAC: Adding query filter params (type=image) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:52.961750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:52.963088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b143d3e-8e6a-4052-b4c0-2992598a6d89 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:52.968518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 523/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:52.974093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:53.035472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:53.081569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdad6cc-d750-4092-bb57-e76114b9fca3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:53.114990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbab423-365f-4b46-8720-eec727672f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:53.116699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbab423-365f-4b46-8720-eec727672f38 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:53.116971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbab423-365f-4b46-8720-eec727672f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:53.117234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbab423-365f-4b46-8720-eec727672f38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:53.117888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbab423-365f-4b46-8720-eec727672f38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:53.119244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbab423-365f-4b46-8720-eec727672f38 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:53.204124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 536/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:04:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:04:53.931000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea340b67-ae9b-45e3-a631-6ca9d1489a16 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:53.931529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea340b67-ae9b-45e3-a631-6ca9d1489a16 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:53.931529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea340b67-ae9b-45e3-a631-6ca9d1489a16 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:53.932957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 524/1060] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:53.937975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:53.938354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:53.938524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:53.957625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:53.958938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:54.031387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 537/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:54.054106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea340b67-ae9b-45e3-a631-6ca9d1489a16 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.067132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:54.158186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea340b67-ae9b-45e3-a631-6ca9d1489a16 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:54.193275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:54.194499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.194778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.195047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.205688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:54.210546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:54.211483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:54.212381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:54.213706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 525/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:54.217933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.277050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:54.345762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a49d937-d07c-4c68-9da5-9b1a4365fbd7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:54.387420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:54.388064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.388344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.388613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.389276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:54.389579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:54.390943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:54.399472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 538/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:54.405096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.477532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:54.510110 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fa3b8-12ed-4e94-8117-5f5f015739c4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:54.543676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:54.544326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.544606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.544877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.556462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:54.557895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-6ce7bde9-77a0-4373-b1f1-c96a0f64175b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:54.571910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:54.573069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:54.573959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:54.573959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:54.573959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:54.573959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 526/1064] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:54.580079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.611132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebca0-db47-4157-a76d-d4ca0b7d160d None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.611416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebca0-db47-4157-a76d-d4ca0b7d160d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.611692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ebca0-db47-4157-a76d-d4ca0b7d160d None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.613087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 527/1065] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:54.618114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.618400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.618677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.637601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:54.638783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:54.685521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:54.690530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57e3b-a4bc-436b-b52a-c4121de7118f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:54.710797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 528/1066] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:54.726438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:54.727044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.727322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.727644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.728267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:54.728573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:54.729936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:54.732467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.742364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 539/1067] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:54.748256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.788766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:54.839793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffa157b-0e85-4f6b-88a1-6a862ae0f848 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:54.856471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd2047-73c7-48b9-82ac-4a412dcecc61 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:54.875882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:54.876602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.876882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.877155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.888853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:54.892431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:54.893061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:54.893339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:54.893611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:54.894070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:54.894960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:54.896735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 529/1068] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:54.901738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.906246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:54.907688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:54.919200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:54.920203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:54.921156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 540/1069] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:54.925874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:54.948033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.020330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7273f087-ca7f-487c-a44c-58306caa30d0 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.031664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6300-95ea-483a-ae49-942965dfa206 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.033672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.055245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.056134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.056414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.056682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.057333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:55.057634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.058991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.066842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 530/1070] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:55.073786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.074072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.074747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.075026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.075296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.076041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:55.076350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.077778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.095670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 541/1071] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:55.101252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.184672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868c996-40e4-46e4-b0e3-339ac8ef2c44 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.185807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.208155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aa183-6a3e-4284-a91b-9a45466ff1bc None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.219787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.220446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.220723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.220992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] PATH_INFO: `/v3/users/2609e262ca9747af8655d92cd5d12aa7` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.222125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] RBAC: Authorizing `identity:get_user(user_id=2609e262ca9747af8655d92cd5d12aa7)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.223546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.225676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 531/1072] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/users/2609e262ca9747af8655d92cd5d12aa7 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:55.231187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.291801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.316390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.317011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.317283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.317546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.327490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.328942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2773e169-6b69-46d1-86c1-99b3041e13d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:55.342756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:55.343743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 542/1073] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdbd44-9444-4e1d-9e99-16d9aba4c34f None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdbd44-9444-4e1d-9e99-16d9aba4c34f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdbd44-9444-4e1d-9e99-16d9aba4c34f None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.350330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 543/1074] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:55.350876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e98bd-3bc7-400d-b7ca-14c32c717345 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.350967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdbd44-9444-4e1d-9e99-16d9aba4c34f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.388334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2afa3e-e234-4fe1-adc1-2ac9376cbc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.388968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2afa3e-e234-4fe1-adc1-2ac9376cbc70 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.389243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2afa3e-e234-4fe1-adc1-2ac9376cbc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.389514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2afa3e-e234-4fe1-adc1-2ac9376cbc70 None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.390649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2afa3e-e234-4fe1-adc1-2ac9376cbc70 None admin] RBAC: Authorizing `identity:get_project(project_id=d1fb6322092b48edb5bd1a86706972f4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.392128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2afa3e-e234-4fe1-adc1-2ac9376cbc70 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.397303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 532/1075] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:55.397303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.397303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.397303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.413278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:55.414472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:55.455196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdbd44-9444-4e1d-9e99-16d9aba4c34f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.480524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 533/1076] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:55.481225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.482592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.492551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.493468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.493901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.494174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.494814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:55.495125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.496513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.508017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 544/1077] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:55.508719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.523140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.578614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94df06-8e95-4d07-b85d-08b09b3340f7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.608598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3ba01-8322-4833-963f-26d323a3f896 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.613641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.614287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.614561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.614832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4/users/2609e262ca9747af8655d92cd5d12aa7/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.616785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fb6322092b48edb5bd1a86706972f4, user_id=2609e262ca9747af8655d92cd5d12aa7, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.618822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.642634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.643236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.643510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.643778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.697864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.710426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.711924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.727410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 534/1078] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:55 2022] PUT /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4/users/2609e262ca9747af8655d92cd5d12aa7/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:04:55.728265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.828958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 545/1079] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:55.830164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.841632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.844580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9750641-c68f-493c-9980-4702b0f49932 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.884311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.884953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.885231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.885499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.892548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.905222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:04:55.905704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:55.906601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:04:55.907507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 535/1080] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:55.913163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:55.932945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:55.946521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf45a4-409b-47f1-bcdb-54abfd1afa3f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:55.981277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:55.981983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:55.982166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:55.982431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:55.993543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:55.994988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:56.010407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:56.011360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:56.013157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:56.013157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:56.013157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:56.013157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 546/1081] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:56.017485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:56.024957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c59e-fb2c-4536-b62c-5dde01e01d07 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:56.063455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:56.064441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:56.064916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:56.065408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:56.066187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:56.066495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:56.067943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:56.081136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 536/1082] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:04:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:56.087492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:56.110849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:56.126834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06585b8-c9cd-4802-93d1-7f4919930df8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:56.163943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:56.164578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:56.164852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:56.165119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:56.165755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:56.166268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:56.167406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9529cbd-e57d-4763-84cc-6e6249951ff2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:56.176854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:56.181148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 547/1083] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:56.204303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53baf7-9b0c-415c-bb19-5d40762e6345 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:56.246052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af164c26-3f3d-479c-8f03-eb3c88b8ed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:56.246732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164c26-3f3d-479c-8f03-eb3c88b8ed95 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:56.247008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164c26-3f3d-479c-8f03-eb3c88b8ed95 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:56.247273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af164c26-3f3d-479c-8f03-eb3c88b8ed95 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/462638e8386c49668290954c0211728c/roles/1f74301298414e66b8138d7093619521` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:56.249492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af164c26-3f3d-479c-8f03-eb3c88b8ed95 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=462638e8386c49668290954c0211728c, role_id=1f74301298414e66b8138d7093619521)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:56.250959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af164c26-3f3d-479c-8f03-eb3c88b8ed95 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:56.368135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 537/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:04:56 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/462638e8386c49668290954c0211728c/roles/1f74301298414e66b8138d7093619521 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:04:57.763847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ece89-63bb-4453-8533-d79a0077e270 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:57.764381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ece89-63bb-4453-8533-d79a0077e270 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:57.764381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ece89-63bb-4453-8533-d79a0077e270 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:57.765665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 548/1085] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:04:57.770822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:57.771249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:57.771472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:57.789486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:57.790658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:57.855831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 538/1086] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:04:57.879140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ece89-63bb-4453-8533-d79a0077e270 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:57.978987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ece89-63bb-4453-8533-d79a0077e270 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.014483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.015027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.015303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.015573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.025440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:58.030097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:58.031234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:58.032516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 549/1087] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:04:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:58.037463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.050000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.145691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf80b26-e660-40fb-b062-f220aad13cee None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.181925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.182558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.182835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.183101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.183764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:58.184098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:58.185459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:58.194198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 539/1088] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:58.200021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.262584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.303293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4eecc-00eb-4ce0-b7c7-9fc0de4d609a None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.338532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d7d617-35bb-4888-aaa0-7d311a9da4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.339230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7d617-35bb-4888-aaa0-7d311a9da4ea None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.339506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7d617-35bb-4888-aaa0-7d311a9da4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.339775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d7d617-35bb-4888-aaa0-7d311a9da4ea None admin] PATH_INFO: `/v3/users/2609e262ca9747af8655d92cd5d12aa7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.340923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7d617-35bb-4888-aaa0-7d311a9da4ea None admin] RBAC: Authorizing `identity:get_user(user_id=2609e262ca9747af8655d92cd5d12aa7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:58.342337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d7d617-35bb-4888-aaa0-7d311a9da4ea None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:58.343913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 550/1089] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/users/2609e262ca9747af8655d92cd5d12aa7 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:58.350244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.425558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.456527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4195c32-e10d-4f7e-b50c-c9b63c0d68f4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.492673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.493283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.493559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.493829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.501006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25d47-4b9d-4149-8410-4c4dc2c96ec4 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.501281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25d47-4b9d-4149-8410-4c4dc2c96ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.501546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c25d47-4b9d-4149-8410-4c4dc2c96ec4 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.504021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 551/1090] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:04:58.509885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.510168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.510439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.531451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:04:58.532679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:04:58.566831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:58.568405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:58.619320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 552/1091] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:04:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:04:58.638198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.642976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.669513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 540/1092] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 10 11:04:58.675384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.707596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.753435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb075c8-4803-4092-bb3c-8a3afeadb7ad None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.777274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532b22b-c2dd-4bdd-be5d-7a31dcb9f9dc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.790258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.790885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.791157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.791531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.804233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:04:58.809969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:58.810963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:58.812265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 553/1093] 10.222.0.13 () {56 vars in 1120 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:58.814446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.815148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.815504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.815840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.817125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] RBAC: Authorizing `identity:get_project(project_id=d1fb6322092b48edb5bd1a86706972f4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:58.817735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.818926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:58.824161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 541/1094] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:58.826594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.870090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.926866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1176e-ea04-44bd-be93-b8b8724c2fe3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.932919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81477d65-215a-4fef-86dd-85fd33621d15 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:58.946206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:58.963454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.964316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.964594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.964866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.965516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:58.965813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:58.967175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:58.972506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:58.973111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:58.973391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:58.973661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:58.974526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=d1fb6322092b48edb5bd1a86706972f4, user.id=2609e262ca9747af8655d92cd5d12aa7) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:58.974841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:58.976291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:58.977886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 554/1095] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:58.983453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:58.987603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 542/1096] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/role_assignments?user.id=2609e262ca9747af8655d92cd5d12aa7&scope.project.id=d1fb6322092b48edb5bd1a86706972f4&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:04:59.089690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439cca6-7293-40c3-95d6-b9056a894bdf None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:59.125020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:59.125667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:59.125941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:59.126208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:59.137448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:59.138898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:04:59.156097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:59.157076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:04:59.157967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:04:59.157967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:04:59.157967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:59.157967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 555/1097] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:04:59.163378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:59.264727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcde86a-16af-4d75-a99a-c0ccf9e044f3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:59.267067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:59.298081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:59.298682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:59.298958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:59.299223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:59.299852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:59.300188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:59.301524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:59.313686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 543/1098] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:59.319975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:59.350726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:59.421322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452c194-9989-4068-9ee7-c6fdfd8a442b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:59.456043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:59.456668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:59.456943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:59.457212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:59.464391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:59.465824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:04:59.477148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:04:59.483023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:04:59.484079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 556/1099] 10.222.0.13 () {56 vars in 1093 bytes} [Wed Aug 10 11:04:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:04:59.484686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:59.581134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b38af5-6490-43f8-9322-033cc164e9a3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:59.613178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:59.613711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:59.613987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:59.614257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:59.615516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:04:59.615822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:59.617278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:59.634299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 544/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Wed Aug 10 11:04:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:04:59.639964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:59.729949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:59.738243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bb247-061f-47b2-8d61-ee6834244288 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:04:59.773008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:04:59.773609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:04:59.773883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:04:59.774150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:04:59.845277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:04:59.846708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:04:59.933237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:04:59.942081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 557/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:04:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:04:59.948012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:04:59.972572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:00.048022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b7b5b-0a47-464b-9186-e7ce7049f034 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:00.080761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:00.081382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:00.081661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:00.081933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:00.092227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:00.093653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:05:00.108620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:00.109768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:05:00.110840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:05:00.110840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:05:00.110840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:05:00.112153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 545/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:05:00.115880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:00.220938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f328bd1-49c3-4b88-b160-ae3acc43fdf7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:00.256040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:00.256660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:00.256930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:00.257195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:00.257820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:00.258110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:00.259461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:00.270257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 558/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:00.276408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:00.349466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:00.373797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097068f-4291-4dc5-a7d6-8096b9c08a24 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:00.406627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:00.407233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:00.407505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:00.407772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:00.416918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:00.418327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:05:00.429728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:00.431018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:05:00.431905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 546/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:00.436843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:00.487472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:00.533689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ade08-f722-44de-b44d-c9101c5c7eb7 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:00.567622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:00.568275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:00.568598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:00.568897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:00.569578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:00.569880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:00.571298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3906a204-63fa-4f04-a89d-d1c0f44d437b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:00.583051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 559/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:00.589458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:00.686505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e5e9e-58ac-4ba8-9f72-4ed89469af9a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:00.718937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:00.719530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:00.719805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:00.720098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:00.720799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] RBAC: Adding query filter params (role.id=1f74301298414e66b8138d7093619521, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=462638e8386c49668290954c0211728c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:00.721101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:00.722510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab2d91-ae4b-4558-9d85-5abc79fdcc2a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:00.724226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:00.729889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 547/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:00 2022] GET /identity/v3/role_assignments?user.id=462638e8386c49668290954c0211728c&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=1f74301298414e66b8138d7093619521 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:01.057141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc934cc-39b6-4033-adc3-5a6cf4422fcd None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:01.057666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc934cc-39b6-4033-adc3-5a6cf4422fcd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:01.057666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc934cc-39b6-4033-adc3-5a6cf4422fcd None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:01.059005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 560/1107] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:01.064362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:01.064780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:01.064913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:01.083411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:01.085431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:01.154990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 548/1108] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:01.179979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc934cc-39b6-4033-adc3-5a6cf4422fcd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:01.284613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc934cc-39b6-4033-adc3-5a6cf4422fcd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:01.298254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:01.320130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f717fb-074f-4c24-89a8-541f703d2775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:01.320769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f717fb-074f-4c24-89a8-541f703d2775 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:01.321048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f717fb-074f-4c24-89a8-541f703d2775 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:01.321319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f717fb-074f-4c24-89a8-541f703d2775 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:01.322452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f717fb-074f-4c24-89a8-541f703d2775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:01.323869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f717fb-074f-4c24-89a8-541f703d2775 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:01.325468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 561/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:05:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:01.331437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:01.436441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10638ac0-3a10-487c-b946-ed5c6b69d9f4 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:01.464353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:01.472051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd2989-89d4-4e2f-b0f0-9bbd232c1cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:01.476142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd2989-89d4-4e2f-b0f0-9bbd232c1cef None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:01.476142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd2989-89d4-4e2f-b0f0-9bbd232c1cef None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:01.476142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd2989-89d4-4e2f-b0f0-9bbd232c1cef None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:01.476142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd2989-89d4-4e2f-b0f0-9bbd232c1cef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:01.476142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd2989-89d4-4e2f-b0f0-9bbd232c1cef None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:01.579255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 549/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:05:01 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:05:03.591252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb18d7-3839-4713-a5d2-75468025deaf None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:03.591792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb18d7-3839-4713-a5d2-75468025deaf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:03.591917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fb18d7-3839-4713-a5d2-75468025deaf None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:03.593222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 562/1111] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:03.598209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:03.598599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:03.598753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:03.618126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:03.619303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:03.684015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 550/1112] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:03.706819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb18d7-3839-4713-a5d2-75468025deaf None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:03.805411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fb18d7-3839-4713-a5d2-75468025deaf None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:03.838927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:03.839542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:03.839822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:03.840122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] PATH_INFO: `/v3/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:03.841517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] RBAC: Authorizing `identity:get_role(role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:03.842882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:03.844223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 563/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:03 2022] GET /identity/v3/roles/22011eb257084dc58e55ad842a3a3292 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:03.850341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:03.946394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677d533-ab95-4dd7-8c88-db2f9a1a286a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:03.960598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:03.980259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:03.980876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:03.981157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:03.981430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] PATH_INFO: `/v3/users/28dcb9020e52477299ecd5c6853d2c56` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:03.994128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] RBAC: Authorizing `identity:get_user(user_id=28dcb9020e52477299ecd5c6853d2c56)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:03.995551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:03.997160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 551/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:03 2022] GET /identity/v3/users/28dcb9020e52477299ecd5c6853d2c56 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:05:04.003008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:04.050212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:04.100405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482fc514-e764-4a16-a6a5-40ed39839fc2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:04.131893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:04.132518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:04.132778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:04.133044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:04.196597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:04.197956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:04.243192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:04.294585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 564/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:04.300565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:04.399727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfeb64c-9a3e-42c9-882e-f8e9f2ccd938 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:04.433018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f24c34c-7778-4145-b8ca-c98b8e9623c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:04.433637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24c34c-7778-4145-b8ca-c98b8e9623c8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:04.433911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24c34c-7778-4145-b8ca-c98b8e9623c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:04.434178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f24c34c-7778-4145-b8ca-c98b8e9623c8 None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:04.435264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24c34c-7778-4145-b8ca-c98b8e9623c8 None admin] RBAC: Authorizing `identity:get_project(project_id=d1fb6322092b48edb5bd1a86706972f4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:04.436700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f24c34c-7778-4145-b8ca-c98b8e9623c8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:04.438283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 552/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:04 2022] GET /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:04.445596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:04.540401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8fe77-1d06-4e70-add2-042b04d86143 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:04.573761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:04.574346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:04.574619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:04.574886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:04.575590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] RBAC: Adding query filter params (role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=d1fb6322092b48edb5bd1a86706972f4, user.id=28dcb9020e52477299ecd5c6853d2c56) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:04.575892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:04.577338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c810e07-3039-4623-b4bf-7b1dbe5ca4f5 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:04.582406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 565/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:04 2022] GET /identity/v3/role_assignments?user.id=28dcb9020e52477299ecd5c6853d2c56&scope.project.id=d1fb6322092b48edb5bd1a86706972f4&role.id=22011eb257084dc58e55ad842a3a3292 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:06.527172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c4f5-99ec-4307-82a8-c7698ca915e1 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:06.527713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c4f5-99ec-4307-82a8-c7698ca915e1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:06.527713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0c4f5-99ec-4307-82a8-c7698ca915e1 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:06.529076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 553/1118] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:06.534204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:06.534652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:06.534749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:06.552810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:06.553991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:06.561379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:06.618300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 566/1119] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:06.640674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c4f5-99ec-4307-82a8-c7698ca915e1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:06.682459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:06.738778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0c4f5-99ec-4307-82a8-c7698ca915e1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:06.772113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:06.772726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:06.773005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:06.773278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] PATH_INFO: `/v3/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:06.774655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] RBAC: Authorizing `identity:get_role(role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:06.776047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:06.777341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 554/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:06 2022] GET /identity/v3/roles/22011eb257084dc58e55ad842a3a3292 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:06.783080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:06.878186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe779e11-b8d6-49a7-9f70-ce02691dc812 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:06.910832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:06.911388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:06.911689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:06.911959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] PATH_INFO: `/v3/users/28dcb9020e52477299ecd5c6853d2c56` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:06.913088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] RBAC: Authorizing `identity:get_user(user_id=28dcb9020e52477299ecd5c6853d2c56)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:06.914495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:06.916057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 567/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:06 2022] GET /identity/v3/users/28dcb9020e52477299ecd5c6853d2c56 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:06.921706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:07.017700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2d3641-ea6f-4ff3-8cc5-277797fc1154 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:07.026009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:07.049747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4e8d9-3620-41a2-8eb1-63b94e4f3a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:07.050353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4e8d9-3620-41a2-8eb1-63b94e4f3a9c None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:07.050627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4e8d9-3620-41a2-8eb1-63b94e4f3a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:07.050892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4e8d9-3620-41a2-8eb1-63b94e4f3a9c None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:07.051983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4e8d9-3620-41a2-8eb1-63b94e4f3a9c None admin] RBAC: Authorizing `identity:get_project(project_id=d1fb6322092b48edb5bd1a86706972f4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:07.054138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4e8d9-3620-41a2-8eb1-63b94e4f3a9c None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:07.055712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 555/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:06 2022] GET /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:07.062107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:07.074951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:07.158392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477378a-8a8f-4410-baea-62aa1d5800cd None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:07.202456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206d4e5-de38-4e18-9a78-9af66bd769e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:07.203036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d4e5-de38-4e18-9a78-9af66bd769e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:07.203311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d4e5-de38-4e18-9a78-9af66bd769e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:07.203581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206d4e5-de38-4e18-9a78-9af66bd769e9 None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4/users/28dcb9020e52477299ecd5c6853d2c56/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:07.208842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d4e5-de38-4e18-9a78-9af66bd769e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d1fb6322092b48edb5bd1a86706972f4, user_id=28dcb9020e52477299ecd5c6853d2c56, role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:07.210258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206d4e5-de38-4e18-9a78-9af66bd769e9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:07.303815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 568/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:05:07 2022] PUT /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4/users/28dcb9020e52477299ecd5c6853d2c56/roles/22011eb257084dc58e55ad842a3a3292 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:05:09.327528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba310b-aae3-46c6-9eb3-a6f934bb1c17 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:09.328120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba310b-aae3-46c6-9eb3-a6f934bb1c17 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:09.328120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba310b-aae3-46c6-9eb3-a6f934bb1c17 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:09.329404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 556/1124] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:09.334529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:09.334971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:09.335072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:09.352489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:09.353653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:09.416379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:09.420799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 569/1125] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:09.444757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba310b-aae3-46c6-9eb3-a6f934bb1c17 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:09.545013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba310b-aae3-46c6-9eb3-a6f934bb1c17 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:09.578557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:09.579101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:09.579375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:09.579641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] PATH_INFO: `/v3/roles/22011eb257084dc58e55ad842a3a3292` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:09.581006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] RBAC: Authorizing `identity:get_role(role_id=22011eb257084dc58e55ad842a3a3292)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:09.582347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:09.584714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 557/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:09 2022] GET /identity/v3/roles/22011eb257084dc58e55ad842a3a3292 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:09.590489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:09.683977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2340-ebcc-4301-8694-75f01f372f45 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:09.717558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:09.718164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:09.718439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:09.718707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] PATH_INFO: `/v3/users/28dcb9020e52477299ecd5c6853d2c56` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:09.719805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] RBAC: Authorizing `identity:get_user(user_id=28dcb9020e52477299ecd5c6853d2c56)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:09.721237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:09.722771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 570/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:09 2022] GET /identity/v3/users/28dcb9020e52477299ecd5c6853d2c56 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:09.728643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:09.773932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:09.826693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d6b21-7155-4c3e-9201-6e15c602e200 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:09.858024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:09.858536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:09.858814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:09.859080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:09.922325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:09.923733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:09.966099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:10.018490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 558/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:10.024637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:10.088362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:10.119880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b2203-e6a4-4b78-b77b-7dc430a8bf7f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:10.151725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d01ee-fd7d-4bdd-94bb-f8f5107474bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:10.152365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d01ee-fd7d-4bdd-94bb-f8f5107474bc None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:10.152639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d01ee-fd7d-4bdd-94bb-f8f5107474bc None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:10.152905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d01ee-fd7d-4bdd-94bb-f8f5107474bc None admin] PATH_INFO: `/v3/projects/d1fb6322092b48edb5bd1a86706972f4` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:10.153981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d01ee-fd7d-4bdd-94bb-f8f5107474bc None admin] RBAC: Authorizing `identity:get_project(project_id=d1fb6322092b48edb5bd1a86706972f4)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:10.155381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d01ee-fd7d-4bdd-94bb-f8f5107474bc None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:10.156992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 571/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:10 2022] GET /identity/v3/projects/d1fb6322092b48edb5bd1a86706972f4 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:10.163587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:10.259201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef48120-7615-4e5e-8a50-63b3d71876e4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:10.291777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:10.292383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:10.292660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:10.292930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:10.293636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] RBAC: Adding query filter params (role.id=22011eb257084dc58e55ad842a3a3292, scope.project.id=d1fb6322092b48edb5bd1a86706972f4, user.id=28dcb9020e52477299ecd5c6853d2c56) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:10.293940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:10.295344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b3f67c-3862-43d2-9e65-cac66b236df2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:10.300838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 559/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:10 2022] GET /identity/v3/role_assignments?user.id=28dcb9020e52477299ecd5c6853d2c56&scope.project.id=d1fb6322092b48edb5bd1a86706972f4&role.id=22011eb257084dc58e55ad842a3a3292 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:05:12.290407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a01fb7-403c-427e-85bb-d4a684c4e4c3 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:12.290938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a01fb7-403c-427e-85bb-d4a684c4e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:12.290938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a01fb7-403c-427e-85bb-d4a684c4e4c3 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:12.292312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 572/1131] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:12.297618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:12.298032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:12.298173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:12.316183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:12.317362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:12.319374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:12.384016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 560/1132] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:12.407203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a01fb7-403c-427e-85bb-d4a684c4e4c3 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:12.504991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a01fb7-403c-427e-85bb-d4a684c4e4c3 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:12.538236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f2730-a176-4d4c-9d20-2909a6296251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:12.538853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f2730-a176-4d4c-9d20-2909a6296251 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:12.539129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f2730-a176-4d4c-9d20-2909a6296251 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:12.539397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f2730-a176-4d4c-9d20-2909a6296251 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:12.540579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f2730-a176-4d4c-9d20-2909a6296251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:12.541940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f2730-a176-4d4c-9d20-2909a6296251 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:12.543495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 573/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:05:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:12.549355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:12.644356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a28f2c-db6e-4f23-9bd9-d6824da7aa1e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:12.677671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e46b64-9e58-4dab-a8b6-0565f921d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:12.678253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46b64-9e58-4dab-a8b6-0565f921d37d None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:12.678533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46b64-9e58-4dab-a8b6-0565f921d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:12.678801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e46b64-9e58-4dab-a8b6-0565f921d37d None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:12.679428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e46b64-9e58-4dab-a8b6-0565f921d37d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:12.681131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e46b64-9e58-4dab-a8b6-0565f921d37d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:12.689992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:12.787693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 561/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:05:12 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:05:14.796493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fda059-a114-4831-be8f-036e020e3611 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:14.796993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fda059-a114-4831-be8f-036e020e3611 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:14.796993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fda059-a114-4831-be8f-036e020e3611 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:14.798312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 574/1135] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:14.805254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:14.805680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:14.805789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:14.823736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:14.824945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:14.888532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 562/1136] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:14.915823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fda059-a114-4831-be8f-036e020e3611 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:14.919625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:15.015220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fda059-a114-4831-be8f-036e020e3611 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:15.047935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa9e1b-68a9-4337-a818-cf1c16f531a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:15.048513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa9e1b-68a9-4337-a818-cf1c16f531a6 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:15.048786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa9e1b-68a9-4337-a818-cf1c16f531a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:15.049050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa9e1b-68a9-4337-a818-cf1c16f531a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:15.050140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa9e1b-68a9-4337-a818-cf1c16f531a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:15.051544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa9e1b-68a9-4337-a818-cf1c16f531a6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:15.053373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 575/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:15.059390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:15.153685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aa40b-158d-4c0f-913a-dc9f5d0a0b24 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:15.173108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:15.186716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960daa55-bc62-4a11-87c8-ec2f6f8c80d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:15.187303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960daa55-bc62-4a11-87c8-ec2f6f8c80d6 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:15.187580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960daa55-bc62-4a11-87c8-ec2f6f8c80d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:15.187849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960daa55-bc62-4a11-87c8-ec2f6f8c80d6 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:15.188504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960daa55-bc62-4a11-87c8-ec2f6f8c80d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:15.189972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960daa55-bc62-4a11-87c8-ec2f6f8c80d6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:15.459314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 563/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:05:15 2022] POST /identity/v3/users => generated 286 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:05:17.563589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f4381-9891-4834-a699-e2a4b1a01a23 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:17.564145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f4381-9891-4834-a699-e2a4b1a01a23 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:17.564259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5f4381-9891-4834-a699-e2a4b1a01a23 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:17.565515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 576/1139] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:17.571243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:17.571624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:17.571782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:17.589544 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:17.590717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:17.654450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 564/1140] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:17.678425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f4381-9891-4834-a699-e2a4b1a01a23 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:17.746898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:17.775705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5f4381-9891-4834-a699-e2a4b1a01a23 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:17.810738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:17.811354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:17.811627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:17.811894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:17.821840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:05:17.826475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:17.827603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:05:17.828697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 577/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:05:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:17.834541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:17.934811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4c95f0-5bda-4a4a-a350-3da9a2c86a62 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:17.942334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:17.967661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:17.968283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:17.968561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:17.968831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:17.969487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:17.969787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:17.971140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:17.979946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 565/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:05:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:17.985830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:18.084625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736efc4d-ab2e-42d0-82ad-146d1bd90508 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:18.112603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:18.117660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:18.118272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:18.118544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:18.118809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] PATH_INFO: `/v3/users/5f86a299464d475ab5dc8664a4f8f49c` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:18.131444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] RBAC: Authorizing `identity:get_user(user_id=5f86a299464d475ab5dc8664a4f8f49c)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:18.132928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:18.134470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 578/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:17 2022] GET /identity/v3/users/5f86a299464d475ab5dc8664a4f8f49c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:18.140381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:18.240490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21c5c14-1ecc-46b3-9aee-0753a700547b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:18.277454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:18.278047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:18.278320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:18.278587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:18.312725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:18.344872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:18.346296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:18.441876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 566/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:18.448626 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:18.543754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ed3ad-67a7-48a5-a874-1cd276e52ab8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:18.576165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42424031-678f-4602-9fb2-5c73def8074c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:18.576771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42424031-678f-4602-9fb2-5c73def8074c None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:18.577042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42424031-678f-4602-9fb2-5c73def8074c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:18.577306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42424031-678f-4602-9fb2-5c73def8074c None admin] PATH_INFO: `/v3/projects/783ffb96b945443eb0331c21eb139883` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:18.578383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42424031-678f-4602-9fb2-5c73def8074c None admin] RBAC: Authorizing `identity:get_project(project_id=783ffb96b945443eb0331c21eb139883)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:18.579783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42424031-678f-4602-9fb2-5c73def8074c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:18.581385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 579/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:18 2022] GET /identity/v3/projects/783ffb96b945443eb0331c21eb139883 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:18.588013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:18.651800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:18.683382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fd582-f871-4ed4-a757-d949bfdd44bb None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:18.716579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:18.717165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:18.717448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:18.717722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:18.718433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=783ffb96b945443eb0331c21eb139883, user.id=5f86a299464d475ab5dc8664a4f8f49c) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:18.718740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:18.720325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f02f22e-ddfb-4da5-8ce9-faf1ea4aba1a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:18.725588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 567/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:18 2022] GET /identity/v3/role_assignments?user.id=5f86a299464d475ab5dc8664a4f8f49c&scope.project.id=783ffb96b945443eb0331c21eb139883&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:20.697374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec576e05-26ff-4e3d-9832-27fc89dc1e95 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.697917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec576e05-26ff-4e3d-9832-27fc89dc1e95 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.697917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec576e05-26ff-4e3d-9832-27fc89dc1e95 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.699164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 580/1147] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:20.704340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.704775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.704887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.723508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:20.725081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:20.789512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 568/1148] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:20.811439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec576e05-26ff-4e3d-9832-27fc89dc1e95 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:20.869201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:20.910314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec576e05-26ff-4e3d-9832-27fc89dc1e95 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:20.942267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:20.943507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:20.943778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:20.944071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:20.954239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:05:20.959501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:20.960669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:05:20.961694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 581/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:05:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:20.966536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:21.025735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:21.062803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bf1bc5-04cd-4a08-8b7c-6557cf2f7e61 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:21.095662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:21.096297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:21.096579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:21.096852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:21.097505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:21.097805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:21.099157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:21.107309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 569/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:05:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:21.113427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:21.211562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bf9da-92c7-4fbf-af94-1792480d26c4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:21.227890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:21.243677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:21.244312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:21.244586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:21.244852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] PATH_INFO: `/v3/users/5f86a299464d475ab5dc8664a4f8f49c` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:21.245942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] RBAC: Authorizing `identity:get_user(user_id=5f86a299464d475ab5dc8664a4f8f49c)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:21.247337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:21.248891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 582/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:21 2022] GET /identity/v3/users/5f86a299464d475ab5dc8664a4f8f49c => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:21.254545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:21.353727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a4607-7b85-478a-9e24-413f300689a3 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:21.376265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:21.387401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e695086-a0d7-4821-bb9a-25b58c5d6df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:21.399054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e695086-a0d7-4821-bb9a-25b58c5d6df8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:21.399054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e695086-a0d7-4821-bb9a-25b58c5d6df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:21.399054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e695086-a0d7-4821-bb9a-25b58c5d6df8 None admin] PATH_INFO: `/v3/projects/783ffb96b945443eb0331c21eb139883` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:21.399054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e695086-a0d7-4821-bb9a-25b58c5d6df8 None admin] RBAC: Authorizing `identity:get_project(project_id=783ffb96b945443eb0331c21eb139883)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:21.399054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e695086-a0d7-4821-bb9a-25b58c5d6df8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:21.399054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 570/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:21 2022] GET /identity/v3/projects/783ffb96b945443eb0331c21eb139883 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:21.399633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:21.495457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c08ea9-2eda-4457-8177-874fef1fab32 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:21.527883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7adae9-5a62-4426-b107-aece16a6b320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:21.528550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7adae9-5a62-4426-b107-aece16a6b320 None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:21.528830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7adae9-5a62-4426-b107-aece16a6b320 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:21.529099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7adae9-5a62-4426-b107-aece16a6b320 None admin] PATH_INFO: `/v3/projects/783ffb96b945443eb0331c21eb139883/users/5f86a299464d475ab5dc8664a4f8f49c/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:21.530976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7adae9-5a62-4426-b107-aece16a6b320 None admin] RBAC: Authorizing `identity:create_grant(project_id=783ffb96b945443eb0331c21eb139883, user_id=5f86a299464d475ab5dc8664a4f8f49c, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:21.532447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7adae9-5a62-4426-b107-aece16a6b320 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:21.606694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 583/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:05:21 2022] PUT /identity/v3/projects/783ffb96b945443eb0331c21eb139883/users/5f86a299464d475ab5dc8664a4f8f49c/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:05:23.596849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c53a6b-d32c-4e5a-a1ae-561b06757561 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:23.597390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c53a6b-d32c-4e5a-a1ae-561b06757561 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:23.597390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c53a6b-d32c-4e5a-a1ae-561b06757561 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:23.598727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 571/1154] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:23.604525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:23.604932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:23.605055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:23.624116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:23.625228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:23.665726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:23.689022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 584/1155] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:23.712047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c53a6b-d32c-4e5a-a1ae-561b06757561 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:23.811352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c53a6b-d32c-4e5a-a1ae-561b06757561 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:23.844763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:23.845374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:23.845656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:23.845929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:23.856225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:05:23.860895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:23.861996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:05:23.863177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 572/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:23.868531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:23.964077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7fcd3-e9a3-49f2-858c-1a5a13272ab5 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:23.996279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:23.996874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:23.997150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:23.997419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:23.998098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:23.998397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:23.999741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:24.008210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 585/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:24.013898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:24.065067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:24.110092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5f199-103d-4df7-8c03-e25fc9e944c1 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:24.142715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:24.143332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.143610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.143882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] PATH_INFO: `/v3/users/5f86a299464d475ab5dc8664a4f8f49c` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.145018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] RBAC: Authorizing `identity:get_user(user_id=5f86a299464d475ab5dc8664a4f8f49c)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:24.146526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:24.148084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 573/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:24 2022] GET /identity/v3/users/5f86a299464d475ab5dc8664a4f8f49c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:24.153781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:24.192113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:24.248546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234faf0-12e2-4424-bd45-00a9c532394b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:24.280926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:24.281518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.281792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.282058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.350846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:24.352325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:24.390118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:24.448766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 586/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:24.454774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:24.550530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e8793-5d17-4e67-8115-fa387f1ed81b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:24.570476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:24.584011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a6e34-06a8-447d-a034-dc0ed192ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:24.586609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a6e34-06a8-447d-a034-dc0ed192ac14 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.586609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a6e34-06a8-447d-a034-dc0ed192ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.586609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a6e34-06a8-447d-a034-dc0ed192ac14 None admin] PATH_INFO: `/v3/projects/783ffb96b945443eb0331c21eb139883` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.586609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a6e34-06a8-447d-a034-dc0ed192ac14 None admin] RBAC: Authorizing `identity:get_project(project_id=783ffb96b945443eb0331c21eb139883)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:24.587618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13a6e34-06a8-447d-a034-dc0ed192ac14 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:24.589234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 574/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:24 2022] GET /identity/v3/projects/783ffb96b945443eb0331c21eb139883 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:24.595352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:24.691640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ada99-505c-4bff-965f-bb32b40a7e08 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:24.723502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:24.724940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:24.725217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:24.725482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:24.726183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=783ffb96b945443eb0331c21eb139883, user.id=5f86a299464d475ab5dc8664a4f8f49c) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:24.726484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:24.727889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0eb66f-06d7-43ec-ba40-c48eeb4a8a6a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:24.733345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 587/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:24 2022] GET /identity/v3/role_assignments?user.id=5f86a299464d475ab5dc8664a4f8f49c&scope.project.id=783ffb96b945443eb0331c21eb139883&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:26.768131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf3775-04be-47e6-be8a-1332219805e4 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:26.768657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf3775-04be-47e6-be8a-1332219805e4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:26.768657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf3775-04be-47e6-be8a-1332219805e4 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:26.769918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 575/1162] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:26.775546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:26.775929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:26.776134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:26.782324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:26.794667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:26.795847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:26.861450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 588/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:26.884338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf3775-04be-47e6-be8a-1332219805e4 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:26.983501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf3775-04be-47e6-be8a-1332219805e4 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:27.016457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:27.017034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:27.017306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:27.017573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:27.071449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:27.082884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:27.085486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:27.180617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 576/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:27.186721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:27.281231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6122dcd-5c65-4c6a-ae62-5c5247adea28 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:27.310151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:27.314907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:27.315523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:27.315802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:27.316103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:27.323078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:27.324515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-37305db2-2645-48a5-b2b5-cf9c938612f1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:27.336236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:27.337492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:05:27.338425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 589/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:05:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:27.343298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:27.429017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:27.440446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d4037-9f8e-4ac8-9552-007fa3e42e9f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:27.473800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:27.474438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:27.474717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:27.474989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:27.475619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:27.475920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:27.477293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc14fa20-62cb-499d-9076-d7269e997a21 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:27.484065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 577/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Wed Aug 10 11:05:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:29.470339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2dfa-fe39-4f25-9efa-6531f934cc15 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:29.470855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2dfa-fe39-4f25-9efa-6531f934cc15 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:29.470969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2dfa-fe39-4f25-9efa-6531f934cc15 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:29.472249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 590/1167] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:29.477782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071c522-76a2-40ea-a4d9-a2099d11b6f4 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:29.478153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071c522-76a2-40ea-a4d9-a2099d11b6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:29.478331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071c522-76a2-40ea-a4d9-a2099d11b6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:29.495845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e071c522-76a2-40ea-a4d9-a2099d11b6f4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:29.497050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e071c522-76a2-40ea-a4d9-a2099d11b6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:29.561127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 578/1168] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:29.583635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2dfa-fe39-4f25-9efa-6531f934cc15 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:29.682523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2dfa-fe39-4f25-9efa-6531f934cc15 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:29.715690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d8bf5-e12c-47e1-af89-b62ab0a4470c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:29.716300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d8bf5-e12c-47e1-af89-b62ab0a4470c None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:29.716579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d8bf5-e12c-47e1-af89-b62ab0a4470c None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:29.716848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d8bf5-e12c-47e1-af89-b62ab0a4470c None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:29.717499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d8bf5-e12c-47e1-af89-b62ab0a4470c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:29.718852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d8bf5-e12c-47e1-af89-b62ab0a4470c None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:29.796137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 591/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Wed Aug 10 11:05:29 2022] POST /identity/v3/domains => generated 267 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:05:31.803779 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6553f9-dc21-415e-aa5b-7854216b0352 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:31.804325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6553f9-dc21-415e-aa5b-7854216b0352 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:31.804325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6553f9-dc21-415e-aa5b-7854216b0352 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:31.805595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 579/1170] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:31.810896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:31.811341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:31.811444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:31.829488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:31.830647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:31.875523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:31.894849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 592/1171] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:31.918751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6553f9-dc21-415e-aa5b-7854216b0352 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:31.999138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:32.016998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6553f9-dc21-415e-aa5b-7854216b0352 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:32.049417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12129cee-56d6-4116-9a9f-607fbe262cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:32.050029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129cee-56d6-4116-9a9f-607fbe262cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:32.050308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129cee-56d6-4116-9a9f-607fbe262cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:32.050578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12129cee-56d6-4116-9a9f-607fbe262cb7 None admin] PATH_INFO: `/v3/domains/bfe79248f8834b139ce62c104f32e8df` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:32.061512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12129cee-56d6-4116-9a9f-607fbe262cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bfe79248f8834b139ce62c104f32e8df)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:32.062878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12129cee-56d6-4116-9a9f-607fbe262cb7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:32.064481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 580/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Wed Aug 10 11:05:31 2022] GET /identity/v3/domains/bfe79248f8834b139ce62c104f32e8df => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:32.070478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:32.164947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05cc8f3-b777-40da-885f-fb8cc1b75d4b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:32.197509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543466f0-2ba9-455c-847a-0e1265015cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:32.198090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543466f0-2ba9-455c-847a-0e1265015cae None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:32.198363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543466f0-2ba9-455c-847a-0e1265015cae None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:32.198627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543466f0-2ba9-455c-847a-0e1265015cae None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:32.199256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543466f0-2ba9-455c-847a-0e1265015cae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:32.200694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543466f0-2ba9-455c-847a-0e1265015cae None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:32.318900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 593/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:05:32 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:05:34.346892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c8bb8-4d99-4046-8b17-19cfb8345408 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:34.347429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c8bb8-4d99-4046-8b17-19cfb8345408 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:34.347429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c8bb8-4d99-4046-8b17-19cfb8345408 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:34.348855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 581/1174] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:34.354023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:34.354411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:34.354572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:34.372910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:34.374087 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:34.401292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:34.440270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 594/1175] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:34.465948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c8bb8-4d99-4046-8b17-19cfb8345408 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:34.564267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c8bb8-4d99-4046-8b17-19cfb8345408 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:34.597620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ee4a9-a282-438d-b5d4-13c34383a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:34.598175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ee4a9-a282-438d-b5d4-13c34383a746 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:34.598451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ee4a9-a282-438d-b5d4-13c34383a746 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:34.598862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ee4a9-a282-438d-b5d4-13c34383a746 None admin] PATH_INFO: `/v3/domains/bfe79248f8834b139ce62c104f32e8df` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:34.599953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ee4a9-a282-438d-b5d4-13c34383a746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=bfe79248f8834b139ce62c104f32e8df)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:34.601386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ee4a9-a282-438d-b5d4-13c34383a746 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:34.602913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 582/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Wed Aug 10 11:05:34 2022] GET /identity/v3/domains/bfe79248f8834b139ce62c104f32e8df => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:34.609014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:34.704395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ee80-9ccb-4331-92fe-8f5d77034bb1 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:34.736548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079f381-b4dd-4cf5-963d-f361d3d26ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:34.737147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f381-b4dd-4cf5-963d-f361d3d26ff6 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:34.737424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f381-b4dd-4cf5-963d-f361d3d26ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:34.737694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079f381-b4dd-4cf5-963d-f361d3d26ff6 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:34.738333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f381-b4dd-4cf5-963d-f361d3d26ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:34.740324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b079f381-b4dd-4cf5-963d-f361d3d26ff6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:34.811854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 595/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:05:34 2022] POST /identity/v3/users => generated 311 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:05:36.837519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8ae04-3a53-4ba9-8c61-10ae6539040f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:36.838034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8ae04-3a53-4ba9-8c61-10ae6539040f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:36.838248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed8ae04-3a53-4ba9-8c61-10ae6539040f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:36.839598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 583/1178] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:36.844703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:36.845115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:36.845248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:36.863250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:36.864449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:36.877708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:36.928994 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 596/1179] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:36.952514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8ae04-3a53-4ba9-8c61-10ae6539040f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:36.965030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:37.051656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed8ae04-3a53-4ba9-8c61-10ae6539040f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:37.087472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:37.088428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:37.088649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:37.088912 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:37.100788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:05:37.107070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:37.108209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:05:37.109135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 584/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:05:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:37.114468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:37.208696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727dabf-f25a-4655-bff2-7fab363bf868 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:37.240686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:37.241276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:37.241554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:37.241823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:37.242503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:37.242802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:37.244171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:37.251772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 597/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:05:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:37.258124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:37.324959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:37.354303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80f3f9-4d6a-42fb-a80a-cd6e2959df8a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:37.388893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:37.389511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:37.389785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:37.390053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] PATH_INFO: `/v3/users/0e59a2b8c6654dd5b7d39be191002f94` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:37.403460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] RBAC: Authorizing `identity:get_user(user_id=0e59a2b8c6654dd5b7d39be191002f94)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:37.404933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:37.406504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 585/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:37 2022] GET /identity/v3/users/0e59a2b8c6654dd5b7d39be191002f94 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:37.412828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:37.427378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:37.509377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf1cb17-b23d-43be-a99e-9aafa9a4a663 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:37.541457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:37.542109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:37.542380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:37.542643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:37.606387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:37.607740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:37.621789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:37.702094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 598/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:37.708068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:37.806731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7a3a9-190a-4f5a-b033-cae07e2c54a7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:37.832193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:37.840106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f85fdb-2f3f-472b-951e-99b19ebf8330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:37.843960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f85fdb-2f3f-472b-951e-99b19ebf8330 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:37.843960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f85fdb-2f3f-472b-951e-99b19ebf8330 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:37.843960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f85fdb-2f3f-472b-951e-99b19ebf8330 None admin] PATH_INFO: `/v3/projects/65ce77af2bb04b939914431d4913b071` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:37.843960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f85fdb-2f3f-472b-951e-99b19ebf8330 None admin] RBAC: Authorizing `identity:get_project(project_id=65ce77af2bb04b939914431d4913b071)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:37.843960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f85fdb-2f3f-472b-951e-99b19ebf8330 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:37.845482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 586/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:37 2022] GET /identity/v3/projects/65ce77af2bb04b939914431d4913b071 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:37.851591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:37.946330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1dec0-bd86-4f39-96bb-8bfda8bb217f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:37.959311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:37.979564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:37.980180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:37.980457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:37.980722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:37.981426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=65ce77af2bb04b939914431d4913b071, user.id=0e59a2b8c6654dd5b7d39be191002f94) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:37.981731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:37.983127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61245c53-f779-40f6-a04d-2e9f4373cdcc None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:37.988238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 599/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:37 2022] GET /identity/v3/role_assignments?user.id=0e59a2b8c6654dd5b7d39be191002f94&scope.project.id=65ce77af2bb04b939914431d4913b071&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:39.987965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f13a-e30d-4fa8-a86e-8993261e7976 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:39.988551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f13a-e30d-4fa8-a86e-8993261e7976 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:39.988551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3f13a-e30d-4fa8-a86e-8993261e7976 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:39.990586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 587/1186] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:39.995869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:39.996307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:39.996460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:40.014702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:40.015881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:40.079902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 600/1187] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:40.101801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3f13a-e30d-4fa8-a86e-8993261e7976 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:40.200953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3f13a-e30d-4fa8-a86e-8993261e7976 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:40.233477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:40.234087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:40.234366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:40.234638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:40.245349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:05:40.249940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:40.251052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:05:40.252296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 588/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:05:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:40.256957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:40.351636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d1b1b-b566-4cb2-81d8-2225db34eae3 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:40.358721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:40.384855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:40.385459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:40.385737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:40.386008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:40.386662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:40.386962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:40.388347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:40.396363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 601/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:05:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:40.402160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:40.456898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:40.499785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80528774-b65a-44a0-a36d-ebe07a24aa63 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:40.531575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:40.532208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:40.532484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:40.532751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] PATH_INFO: `/v3/users/0e59a2b8c6654dd5b7d39be191002f94` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:40.533841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] RBAC: Authorizing `identity:get_user(user_id=0e59a2b8c6654dd5b7d39be191002f94)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:40.535245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:40.537538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 589/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:40 2022] GET /identity/v3/users/0e59a2b8c6654dd5b7d39be191002f94 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:40.543232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:40.637745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead50d4-c9b1-4653-bfaa-1e9ce44dcb45 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:40.670085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf94a69-17f3-4356-9b0a-b4f7f8526a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:40.670691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94a69-17f3-4356-9b0a-b4f7f8526a86 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:40.670966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94a69-17f3-4356-9b0a-b4f7f8526a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:40.671235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf94a69-17f3-4356-9b0a-b4f7f8526a86 None admin] PATH_INFO: `/v3/projects/65ce77af2bb04b939914431d4913b071` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:40.672360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf94a69-17f3-4356-9b0a-b4f7f8526a86 None admin] RBAC: Authorizing `identity:get_project(project_id=65ce77af2bb04b939914431d4913b071)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:40.673789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf94a69-17f3-4356-9b0a-b4f7f8526a86 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:40.675380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 602/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:40 2022] GET /identity/v3/projects/65ce77af2bb04b939914431d4913b071 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:40.681586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:40.776738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972032c0-6b35-46c8-8e0e-1760cab1ef5e None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:40.799235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:40.809729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c27b8bc-c920-44eb-8966-7d2cac601d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:40.820739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27b8bc-c920-44eb-8966-7d2cac601d16 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:40.820739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27b8bc-c920-44eb-8966-7d2cac601d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:40.820739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c27b8bc-c920-44eb-8966-7d2cac601d16 None admin] PATH_INFO: `/v3/projects/65ce77af2bb04b939914431d4913b071/users/0e59a2b8c6654dd5b7d39be191002f94/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:40.820739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27b8bc-c920-44eb-8966-7d2cac601d16 None admin] RBAC: Authorizing `identity:create_grant(project_id=65ce77af2bb04b939914431d4913b071, user_id=0e59a2b8c6654dd5b7d39be191002f94, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:40.820739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c27b8bc-c920-44eb-8966-7d2cac601d16 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:40.914391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 590/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:05:40 2022] PUT /identity/v3/projects/65ce77af2bb04b939914431d4913b071/users/0e59a2b8c6654dd5b7d39be191002f94/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:05:42.900857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bfcc7-7092-4081-b1ea-935852ac5d62 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:42.901407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bfcc7-7092-4081-b1ea-935852ac5d62 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:42.901407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bfcc7-7092-4081-b1ea-935852ac5d62 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:42.902739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 603/1193] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:05:42.907934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:42.908380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:42.908520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:42.927365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:05:42.928583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:05:42.993239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 591/1194] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:05:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:05:43.016091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bfcc7-7092-4081-b1ea-935852ac5d62 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:43.052813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:43.116136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bfcc7-7092-4081-b1ea-935852ac5d62 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:43.149869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:43.150476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:43.150754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:43.151071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:43.160657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:05:43.165324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:05:43.166389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:05:43.167282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 604/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:05:43.172390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:43.266271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb308147-0c96-4c76-8c01-06dc89972096 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:43.280217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:43.299156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:43.299747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:43.300050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:43.300323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:43.300973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:43.301276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:43.302645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:43.311142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 592/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:43.316392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:43.411129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:43.417065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93b61-6c23-4b63-8e87-418737100543 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:43.450859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:43.451481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:43.451761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:43.452057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] PATH_INFO: `/v3/users/0e59a2b8c6654dd5b7d39be191002f94` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:43.453165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] RBAC: Authorizing `identity:get_user(user_id=0e59a2b8c6654dd5b7d39be191002f94)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:43.454579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:43.456165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 605/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Wed Aug 10 11:05:43 2022] GET /identity/v3/users/0e59a2b8c6654dd5b7d39be191002f94 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:43.462981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:43.559153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e9b4f-a10f-42ac-9308-917eb420b940 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:43.591340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:43.591929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:43.592237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:43.592508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:43.637918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:43.659798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:43.661420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:43.763116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 593/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:05:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:05:43.769068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:43.874909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1520c-d268-4b17-83a1-e146d63183e1 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:43.907727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f28ae6-f37b-4f1a-a76c-23ba08789ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:43.908375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f28ae6-f37b-4f1a-a76c-23ba08789ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:43.908648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f28ae6-f37b-4f1a-a76c-23ba08789ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:43.908914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f28ae6-f37b-4f1a-a76c-23ba08789ce3 None admin] PATH_INFO: `/v3/projects/65ce77af2bb04b939914431d4913b071` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:43.910001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f28ae6-f37b-4f1a-a76c-23ba08789ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=65ce77af2bb04b939914431d4913b071)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:43.912026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f28ae6-f37b-4f1a-a76c-23ba08789ce3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:43.913571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 606/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Wed Aug 10 11:05:43 2022] GET /identity/v3/projects/65ce77af2bb04b939914431d4913b071 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:43.919741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:05:43.994258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:44.015919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963cd6-34f9-4d84-8f41-8faf0271e5fe None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:05:44.049122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:05:44.049716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:05:44.049996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:05:44.050268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:05:44.050988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=65ce77af2bb04b939914431d4913b071, user.id=0e59a2b8c6654dd5b7d39be191002f94) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:05:44.051293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:05:44.053442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbacab-26ec-411a-a606-b57d29c9c4e8 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:05:44.059335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 594/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:05:43 2022] GET /identity/v3/role_assignments?user.id=0e59a2b8c6654dd5b7d39be191002f94&scope.project.id=65ce77af2bb04b939914431d4913b071&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:05:53.904102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=84 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:05:54.056708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=30 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:00.010360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49672c5-2fcc-45f9-b540-4e4f252fe693 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.010952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49672c5-2fcc-45f9-b540-4e4f252fe693 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.010952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49672c5-2fcc-45f9-b540-4e4f252fe693 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.012271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 607/1201] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:00.017288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.017705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.017859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.036186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:00.038628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:00.103662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 595/1202] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:07:00.127739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49672c5-2fcc-45f9-b540-4e4f252fe693 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:00.228304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49672c5-2fcc-45f9-b540-4e4f252fe693 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:00.261559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:00.262160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.262434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.262702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.273240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:00.274741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:00.289623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:00.290740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:00.291919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:00.291919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:00.291919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:00.291919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 608/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:00.297977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:00.397218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b289d3c-17ac-4313-baf1-a318618083ae None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:00.424541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:00.430768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:00.431368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.431645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.431915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.432840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:00.433138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:00.434479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ed5eb-3e19-4ef3-88d4-ea5d1659a85d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:00.445762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 596/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:00.451604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:00.504704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:00.550600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defffff0-c498-4f4e-85e4-a638771dd4a9 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:00.586822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ace84-fb59-4796-b679-a3cbe7da36a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:00.588184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ace84-fb59-4796-b679-a3cbe7da36a6 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:00.588184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ace84-fb59-4796-b679-a3cbe7da36a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:00.588184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ace84-fb59-4796-b679-a3cbe7da36a6 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:00.588755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ace84-fb59-4796-b679-a3cbe7da36a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:00.590155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ace84-fb59-4796-b679-a3cbe7da36a6 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:00.693103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 609/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Wed Aug 10 11:07:00 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:07:02.830288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ddc41-d0ee-498c-9eea-0991d3dc0779 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:02.830857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ddc41-d0ee-498c-9eea-0991d3dc0779 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:02.830857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26ddc41-d0ee-498c-9eea-0991d3dc0779 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:02.832220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 597/1206] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:02.839625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:02.840089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:02.840201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:02.858657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:02.859861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:02.931334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 610/1207] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:02.955007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ddc41-d0ee-498c-9eea-0991d3dc0779 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.054947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26ddc41-d0ee-498c-9eea-0991d3dc0779 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.087978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:03.088557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.088830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.089096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.099221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:07:03.108733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:03.109849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:03.110900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 598/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:07:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:03.112092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.150395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:03.207817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2242f-6a07-4b36-96e2-78db66121572 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.240916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:03.242008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.242281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.242547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.243201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:03.243499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:03.244875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:03.253972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 611/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:03.260218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.310756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:03.358918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9ba2b-66e1-40c3-973f-469eac706ce7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.391195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:03.391816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.392134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.392405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.403080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:03.404525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:03.419210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:03.427455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:03.428760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:03.428760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:03.428760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:03.428760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 599/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:03.429049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.521294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:03.524817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69a9869-67db-4f97-bda8-3f9a4b9b813f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.557106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:03.557955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.558231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.558496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.559118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:03.559413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:03.560783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:03.571541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 612/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:03.577319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.676515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cd75a-819b-4704-b2eb-42e6c820f5d4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.677922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:03.711492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:03.712136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.712417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.712687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.719275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:03.720710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:07:03.731682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:03.733111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:03.734454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 600/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:03.738498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.838481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6128dd41-8618-45e6-9a30-8f9beeaa3f79 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:03.870675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:03.871280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:03.871537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:03.871808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:03.872562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:03.872909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:03.874368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:03.891467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 613/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:03.897981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:03.997668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9f59f-16ad-4a5d-babe-33b08b02927d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:04.029559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.030147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.030418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.030712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.060046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.097141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.098647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.193953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 601/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:07:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:07:04.200255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:04.219711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.301374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92c264-90be-480b-8f71-487d8d8fde1d None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:04.336786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.337447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.337721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.337989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.347653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.349102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:04.363704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:04.364807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:04.365737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:04.365737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:04.365737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:04.365737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 614/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:04.370921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:04.433601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.471129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab133ec2-9978-423f-a904-819607265c2f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:04.503926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.504576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.504850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.505122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.505770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:04.506067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.508445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.520140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 602/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:04.526607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:04.596908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.627855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c5827-c4c7-4eed-99ac-89f050f58911 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:04.659723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.660372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.660651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.660921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.668119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.669597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:07:04.681277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:04.682624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:04.683635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 615/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:07:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:04.688731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:04.787475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb666d-a361-4ed6-81eb-ef291243b34b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:04.810667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.822351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.822951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.823231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.823503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.824227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:04.824560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.825980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d039918f-5f52-451e-974d-4da60ed19ad6 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.837576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 603/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:07:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:07:04.843714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:04.940376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ef6db-8c87-4914-a06f-306bca9acb37 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:04.967664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:04.973326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:04.973922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:04.974196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:04.974462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:04.975813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=8593dfcc1d0542bfbaa6fb169b01ab90) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:04.976114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:04.977532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b48bf3-a8fd-4731-8468-db0ffe815643 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:04.982723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 616/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:07:04 2022] GET /identity/v3/role_assignments?user.id=8593dfcc1d0542bfbaa6fb169b01ab90&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:06.982552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6bc6-07cd-430b-9eb4-5d9777961c59 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:06.983104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6bc6-07cd-430b-9eb4-5d9777961c59 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:06.983198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6bc6-07cd-430b-9eb4-5d9777961c59 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:06.984523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 604/1220] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:06.989829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:06.990280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:06.990373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.008499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:07.009666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:07.074446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 617/1221] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 10 11:07:07.097248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb6bc6-07cd-430b-9eb4-5d9777961c59 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:07.196998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb6bc6-07cd-430b-9eb4-5d9777961c59 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:07.230324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:07.230937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:07.231213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:07.231484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.241552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:07:07.246238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:07.247353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:07.252316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 605/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:07:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:07.253655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:07.354849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d5978-3380-4a0b-b3af-a9a4d182fc69 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:07.378657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:07.389550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:07.395024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:07.395024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:07.395024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.395024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] RBAC: Adding query filter params (name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:07.395024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:07.395024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:07.401931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 618/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:07:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:07.407753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:07.452018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:07.505745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a90807-931a-47c5-bf76-c5f17b432396 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:07.538910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:07.539523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:07.539801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:07.540098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.549581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:07.551000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:07.565119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:07.569508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:07.570568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:07.570568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:07.570568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:07.570568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 606/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:07.572848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:07.671190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f7636-c9ee-4dd2-bbf9-7c50e9b2c005 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:07.704278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:07.704880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:07.705151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:07.705421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.706047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:07.706347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:07.707701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:07.719644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 619/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:07.725512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:07.818840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:07.825715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4112b4c-0405-4821-8dc8-1d5bd6686de5 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:07.858047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:07.858668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:07.858944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:07.859209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:07.866709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:07.867969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:07:07.878941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:07.880094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:07.881273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 607/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:07.886550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:07.933636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:07.983936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e809ad8-a75d-427c-a4bf-c9af54ec31da None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.017503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.018120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.018351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.018616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.019317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:08.019614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.021194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.037403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 620/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 10 11:07:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:08.043191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:08.142948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4792b23-d5ca-4da7-8d1d-53d7c4759438 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.175356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.175969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.176273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.176542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.187669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.192095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:08.203450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:08.204578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:08.205727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:08.205727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:08.205727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:08.205727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 608/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:08.210596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:08.306451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81443b-86c7-4d55-baf2-c30d9e441fa2 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.308996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:08.338776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.339367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.339640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.339905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.340557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:08.340853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.342197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.353598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 621/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:08.361252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:08.371919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:08.460629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c368f-7022-4f49-b9c0-b5b2cbabc22d None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.493194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.493809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.494083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.494350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.500657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.502070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:07:08.513114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:08.514172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:08.515074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 609/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:07:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:08.520205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:08.617838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a6ed9e-66d2-4286-b3b1-45204d238e5f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.650536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.651131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.651407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.651703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.652410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:08.652713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.654124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e1a557-c8ae-4380-bcd8-65c11b0d016a None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.666122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 622/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:07:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:08.672264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:08.736632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:08.769557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10f655-fa24-49a5-81e2-b9ec490e249a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:08.803150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7367de-3f1f-4348-8565-99f48c5f6fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:08.803888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7367de-3f1f-4348-8565-99f48c5f6fb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:08.804194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7367de-3f1f-4348-8565-99f48c5f6fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:08.804466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7367de-3f1f-4348-8565-99f48c5f6fb3 None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/8593dfcc1d0542bfbaa6fb169b01ab90/roles/e39fd30977654c83b4e341ea553b106e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:08.818679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7367de-3f1f-4348-8565-99f48c5f6fb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=8593dfcc1d0542bfbaa6fb169b01ab90, role_id=e39fd30977654c83b4e341ea553b106e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:08.820131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7367de-3f1f-4348-8565-99f48c5f6fb3 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:08.955292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 610/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:07:08 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/8593dfcc1d0542bfbaa6fb169b01ab90/roles/e39fd30977654c83b4e341ea553b106e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:07:10.932225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d549-adc5-447a-9fc4-ce97ee3d93fd None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:10.933047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d549-adc5-447a-9fc4-ce97ee3d93fd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:10.933047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d549-adc5-447a-9fc4-ce97ee3d93fd None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:10.934333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 623/1233] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:10.939493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:10.940922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:10.940922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:10.957869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:10.959036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:11.029390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 611/1234] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:11.053219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d549-adc5-447a-9fc4-ce97ee3d93fd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.078165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.151344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d549-adc5-447a-9fc4-ce97ee3d93fd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.186159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.186716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.186992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.187268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.197972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:07:11.202657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:11.203795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:11.204876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 624/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:11.210973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.255329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.308811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bf55-5121-4eb8-9b7d-84f4c0cf8214 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.343274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.343847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.344283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.344555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.345208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:11.345505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:11.346851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:11.354527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 612/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:11.360550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.446112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.461688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079b5ff-89dd-4d46-81e2-146bae1e078b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.494247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.494882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.495841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.495841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.504926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:11.506455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:11.521351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:11.523237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:11.524378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:11.524378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:11.524378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:11.524378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 625/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:11.530179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.627184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daf9b4-9d4c-4228-ba48-e1245016b9d4 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.636662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.660260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.660853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.661131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.661400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.662027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:11.662325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:11.663840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:11.674985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 613/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:07:11.681391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.779901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6b475-8958-4843-a373-93649b9ccc6b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.812985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.813600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.813876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.814144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.816720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:11.822215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:11.823629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:07:11.834468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:11.835902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:11.839341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 626/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:11.842118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:11.939488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df5990-2797-4890-a8e1-3c1e0c6ee32f None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:11.971803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:11.972431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:11.972708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:11.972978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:11.973690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:11.973994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:11.975940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:11.993156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 614/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:11.999089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:12.096145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d547-5852-4349-880c-ef2945f75159 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:12.129312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:12.129975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.130250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.130517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.191640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:12.198124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:12.199613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:12.293287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 627/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:07:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:07:12.299151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:12.327152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:12.398831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788152a2-6bee-467e-89dd-c8f817094e8a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:12.433053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:12.433681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.433959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.434229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.443809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:12.445364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:12.459192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:12.460788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:12.461869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:12.461869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:12.461869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:12.461869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 615/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 10 11:07:12.468137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:12.560291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:12.565789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f456b3-c450-4fab-b94b-f9d2cb43f52f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:12.598228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:12.598818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.599092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.599355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.599977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:12.603453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:12.603453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:12.613244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 628/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:12.618978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:12.698744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:12.719854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f6efe0-6e95-4061-90b6-8590663df574 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:12.753781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:12.754437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.754751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.755019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.762394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:12.763810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:07:12.775810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:12.776859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:12.778060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 616/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:07:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:12.782896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:12.880482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d807e5-bfa8-447a-89d5-628ae5957bc4 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:12.913555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:12.914149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:12.914479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:12.914756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:12.915437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:12.915734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:12.917176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b97d123-a63e-4193-b174-3b8f7fb9058e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:12.928787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 629/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:07:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:12.935226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:13.032304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dd313-5f0a-44b9-8e7d-ee0a92126841 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:13.066073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:13.066660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:13.066933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:13.067198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:13.067901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] RBAC: Adding query filter params (role.id=e39fd30977654c83b4e341ea553b106e, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=8593dfcc1d0542bfbaa6fb169b01ab90) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:13.068238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:13.070807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a5f04-6325-4925-8861-831bc29b1b3a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:13.076378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 617/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:07:12 2022] GET /identity/v3/role_assignments?user.id=8593dfcc1d0542bfbaa6fb169b01ab90&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e39fd30977654c83b4e341ea553b106e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:15.138344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3228c-eb59-4ce4-963e-d2b6c19d06bb None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.138879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3228c-eb59-4ce4-963e-d2b6c19d06bb None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.138879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3228c-eb59-4ce4-963e-d2b6c19d06bb None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.140243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 630/1247] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:15.145332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.145708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.145889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.152654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:15.164531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:15.165700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:15.229500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 618/1248] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:15.252370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3228c-eb59-4ce4-963e-d2b6c19d06bb None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:15.270773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:15.352281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3228c-eb59-4ce4-963e-d2b6c19d06bb None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:15.386755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:15.387371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.387643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.387907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.398172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:07:15.402734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:15.403831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:15.405222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 631/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:07:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:15.409780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:15.508876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854ba65-f670-4bc6-95c1-b7a9344040f0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:15.536819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:15.542080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:15.542677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.542948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.543215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.543877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:15.544199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:15.545542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:15.553211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 619/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:07:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:15.560022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:15.635261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:15.658475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac33b4e-d8b0-4e4c-ac8b-a62426da5ac0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:15.692379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:15.692993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.693265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.693530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.703584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:15.705020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:15.719604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:15.720655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:15.721575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:15.721575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:15.721575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:15.721575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 632/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:15.727398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:15.830188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31a4f2-b919-4d52-a958-0dbfc5e3bb6b None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:15.863859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:15.864513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:15.864792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:15.865062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:15.865739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:15.866717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:15.868105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:15.879424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 620/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:15.885243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:15.982559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452f812-462f-4460-88c6-847da75a60e3 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:16.009300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:16.017454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:16.018119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:16.018401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:16.018674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:16.026100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:16.027536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:07:16.038990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:16.040046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:16.052171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 633/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:16.052916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:16.103685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:16.148757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d19e22c-856e-4fe6-9dd5-75086e8b6444 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:16.181581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:16.182183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:16.182731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:16.183005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:16.183719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:16.184046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:16.185460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:16.201756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 621/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 10 11:07:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:16.207571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:16.306142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231a9f-a95a-47ab-bede-13a0d970ce33 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:16.339723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:16.340349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:16.340623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:16.340914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:16.394514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:16.406972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:16.408438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:16.502978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 634/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:07:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:07:16.509634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:16.608349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1cee7-70e9-412f-a844-901f87799d2a None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:16.621034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:16.640902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:16.641518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:16.641794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:16.642064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:16.652673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:16.654108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:16.668037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:16.669245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:16.670134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:16.670134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:16.670134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:16.670134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 622/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:16.674906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:16.765967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:16.778280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1aea-fb8b-4d91-bf6b-bef809c4df59 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:16.811430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:16.812036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:16.812313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:16.812580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:16.813240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:16.813535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:16.814869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:16.826598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 635/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:07:16.832437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:16.932048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a29a64-f728-45e9-a115-2fe8b65de1e7 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:16.965093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:16.965710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:16.965991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:16.966261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:16.972790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:16.974214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:07:16.985191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:16.986225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:16.987344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 623/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:16.992974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:17.089250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97d237-e111-4d2e-9d21-d8c7c2adae59 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:17.121902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:17.122525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:17.122819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:17.123074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:17.123754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:17.124108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:17.125524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449eec28-18c8-4698-abfc-55516ca40a73 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:17.137609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 636/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:17.143719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:17.169627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:17.242230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc29684f-2845-4236-bdb5-33e0f4862aff None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:17.275771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:17.276435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:17.276711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:17.276977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:17.277691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=8593dfcc1d0542bfbaa6fb169b01ab90) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:17.277993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:17.279432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548b695-8c2c-447a-9d49-ee83cfe93777 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:17.284911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 624/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:07:17 2022] GET /identity/v3/role_assignments?user.id=8593dfcc1d0542bfbaa6fb169b01ab90&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:19.258419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44e8ed-6fe8-4dd7-a3fe-ecfd7858a4c7 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:19.258968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44e8ed-6fe8-4dd7-a3fe-ecfd7858a4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:19.258968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e44e8ed-6fe8-4dd7-a3fe-ecfd7858a4c7 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:19.260311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 637/1261] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:19.265324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:19.265702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:19.265865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:19.284742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:19.285945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:19.353668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 625/1262] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:19.375932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44e8ed-6fe8-4dd7-a3fe-ecfd7858a4c7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:19.387510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:19.480122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e44e8ed-6fe8-4dd7-a3fe-ecfd7858a4c7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:19.515147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:19.515688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:19.515964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:19.516262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:19.527123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:07:19.531771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:19.540860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:19.541727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 638/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:07:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:19.542601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:19.557804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:19.637802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843379f-8d5e-4e6a-b5d6-fc385aeb34d3 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:19.670513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:19.671108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:19.671382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:19.671649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:19.672327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:19.672625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:19.673962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:19.681729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 626/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:19.687846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:19.754783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:19.788088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56722917-63ea-474b-8080-54bf29cce112 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:19.821930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:19.822620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:19.822956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:19.823235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:19.833518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:19.834998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:19.848672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:19.849793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:19.851090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:19.851090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:19.851090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:19.851090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 639/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:19.856020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:19.937348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:19.956104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84172129-709c-443d-b98e-5af771b6fb22 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:19.987768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:19.988401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:19.988679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:19.988942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:19.989565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:19.989862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:19.991200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.002638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 627/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:20.009062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:20.106708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8fb6f-6a7d-4029-a3e0-be91c50efd52 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:20.125581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:20.139520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:20.153275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.153275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.153275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.153275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:20.153275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:07:20.159620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:20.160820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:20.161932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 640/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:20.167190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:20.265705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6429325-761f-4957-b062-cabe863dc068 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:20.298767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:20.299364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.299643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.299911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.300657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:20.300960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:20.302376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.312086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:20.320460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 628/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 10 11:07:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:20.326304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:20.425755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18df97-a2d1-4987-9f42-908d9da5f996 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:20.458595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:20.459211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.459488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.459757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.469959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:20.471379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:20.485855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:20.486969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:20.488116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:20.488116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:20.488116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:20.488116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 641/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:20.495131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:20.592606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4fd62-7ea2-4e7a-9e19-88fa05ebdf58 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:20.625699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:20.626234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.626509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.626777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.627404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:20.627705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:20.629083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.639782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 629/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:20.645444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:20.677748 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:20.745069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78563cd0-9f8b-48cc-a93d-8131394fd815 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:20.778867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:20.779480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.779757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.780050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.786455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:20.787885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:07:20.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:20.806120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:20.807249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 642/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:20.808193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:20.865281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:20.906530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d08c5fb-c7f8-4229-9212-789fa48d50cb None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:20.938968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:20.939556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:20.939827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:20.940129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:20.940803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:20.941101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:20.942547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc0864-250d-4558-936b-8124133be8f0 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:20.954512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 630/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:07:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:07:20.961270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:21.048439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:21.056807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c11a8c-d9f9-4355-8286-0557baeb279b None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:21.089422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf0dd2c-d73c-4086-898e-ec15377a0a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:21.090060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0dd2c-d73c-4086-898e-ec15377a0a5e None admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:21.090335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0dd2c-d73c-4086-898e-ec15377a0a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:21.090603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf0dd2c-d73c-4086-898e-ec15377a0a5e None admin] PATH_INFO: `/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/8593dfcc1d0542bfbaa6fb169b01ab90/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:21.092556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0dd2c-d73c-4086-898e-ec15377a0a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=ce1741f207fa4b4fb6798f37092a4a10, user_id=8593dfcc1d0542bfbaa6fb169b01ab90, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:21.093968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf0dd2c-d73c-4086-898e-ec15377a0a5e None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:21.202184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 643/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Wed Aug 10 11:07:20 2022] PUT /identity/v3/projects/ce1741f207fa4b4fb6798f37092a4a10/users/8593dfcc1d0542bfbaa6fb169b01ab90/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:07:23.173942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc62f43-52e6-4944-a594-3a103d96b251 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.174542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc62f43-52e6-4944-a594-3a103d96b251 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.174542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc62f43-52e6-4944-a594-3a103d96b251 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.175810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 631/1274] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:23.180958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.181357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.181504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.199920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:23.201117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:23.265262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 644/1275] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:23.289051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc62f43-52e6-4944-a594-3a103d96b251 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:23.395700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc62f43-52e6-4944-a594-3a103d96b251 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:23.413516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:23.429604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:23.430162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.430440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.430708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.440445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 10 11:07:23.445835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:23.450509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:23.452179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 632/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Wed Aug 10 11:07:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:23.453506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:23.530268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:23.547136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce49a2-bb6e-440d-9524-92ecfecb4732 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:23.580941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:23.581528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.581809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.582081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.582730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:23.583030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:23.584419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:23.592541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 645/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Wed Aug 10 11:07:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:07:23.598390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:23.696745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812236a-8ba0-4bcc-b094-d92acef1f784 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:23.729925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:23.730552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.730832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.731103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.740829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:23.742774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:23.756855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:23.758639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:23.759800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:23.759800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:23.759800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:23.759800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 633/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:23.764332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:23.863089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38cb3b-f18b-44d3-80f9-392b11ebc57f None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:23.896270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:23.896864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:23.897140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:23.897409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:23.898100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:23.898398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:23.899744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:23.911029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 646/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:23.917883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:23.949627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:24.016615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b50f0e-b7a9-4566-a428-8b6d9193e7ab None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:24.050177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:24.050796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:24.051074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:24.051371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:24.059185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:24.060647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 10 11:07:24.071943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:24.073183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:24.074344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 634/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:24.078912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:24.082774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:24.175242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c856d-b10d-406c-9338-a70a60ed5bbf None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:24.212697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:24.213338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:24.213503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:24.213772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:24.214527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:24.214832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:24.216334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:24.234067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 647/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Wed Aug 10 11:07:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:24.239902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:24.330669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:24.340514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd41b4-ff9b-4896-ae68-e8949e613469 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:24.375129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:24.375784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:24.376092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:24.376386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:24.444022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:24.445605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:24.531640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:24.542156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 635/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Wed Aug 10 11:07:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 10 11:07:24.548328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:24.608232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:24.645525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ea25e2-5475-4e4f-8f10-3fcc774fd7e8 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:24.677448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:24.678071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:24.678346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:24.678616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:24.688891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:24.690306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 10 11:07:24.705876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:24.714524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 10 11:07:24.715613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 10 11:07:24.715613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 10 11:07:24.715613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:24.715613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 648/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:24.715867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:24.815856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16deb54-507a-4c70-a1bf-2a66e6d22c44 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:24.849935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:24.850534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:24.850815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:24.851086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:24.851864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:24.852200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:24.853559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:24.864856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 636/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Wed Aug 10 11:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:24.870550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:24.969126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcf89e1-ee5f-465a-861f-d5fe4bad5070 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:24.982955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:25.003023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:25.003711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:25.004143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:25.004419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:25.011623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:25.013182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 10 11:07:25.024898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:25.027299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: ERROR keystone.server.flask.application Aug 10 11:07:25.028423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 649/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Wed Aug 10 11:07:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:25.031815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:25.082199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:25.130606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7950c300-cc0f-4dcd-b126-a1e49bcc2ecc None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:25.163327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:25.163971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:25.164784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:25.165054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:25.165729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:25.166030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:25.167440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d8203-54f6-42d8-bd12-20b2945b1726 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:25.179121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 637/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Wed Aug 10 11:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:25.185465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:25.286030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8e4a8-d6ed-4e02-a9cc-3d3a1e6b6db0 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:25.388771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:25.388771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:25.388771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:25.388771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:25.389806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] RBAC: Adding query filter params (role.id=e3983951d6d24c7dadec4aa8c559b5ae, scope.project.id=ce1741f207fa4b4fb6798f37092a4a10, user.id=8593dfcc1d0542bfbaa6fb169b01ab90) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:25.389806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:25.389806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f243e0-0fac-46a1-86c7-cab395e162dd None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:25.389806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 650/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Wed Aug 10 11:07:25 2022] GET /identity/v3/role_assignments?user.id=8593dfcc1d0542bfbaa6fb169b01ab90&scope.project.id=ce1741f207fa4b4fb6798f37092a4a10&role.id=e3983951d6d24c7dadec4aa8c559b5ae => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:27.349476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2ab6-2308-4c66-bf8e-d58a51811c49 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:27.350108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2ab6-2308-4c66-bf8e-d58a51811c49 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:27.350108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d2ab6-2308-4c66-bf8e-d58a51811c49 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:27.351359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 638/1288] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:27.356835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:27.357232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:27.357384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:27.376856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:27.378071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:27.407755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:27.443633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 651/1289] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:27.465807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d2ab6-2308-4c66-bf8e-d58a51811c49 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:27.565349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d2ab6-2308-4c66-bf8e-d58a51811c49 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:27.577201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:27.599368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:27.600065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:27.600305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:27.600578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:27.601149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:27.602522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:27.610990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:27.612072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:27.613001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 639/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:07:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:27.618912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:27.716754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f0fbc-d292-4212-86d8-fadbc6c9cd23 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:27.750129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:27.752965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:27.752965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:27.752965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:27.752965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:27.752965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:27.753529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb2ea8-05a0-4a9b-b79b-03ff4d6a67d3 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:27.758293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 652/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 10 11:07:27 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:27.764118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:27.861058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775d8ec-7add-4e9f-ba2e-809be2b8fbad None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:27.894710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:27.895356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:27.895721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:27.896075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] PATH_INFO: `/v3/services` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:27.896739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:27.897046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:27.898507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf727fc-9ef9-4371-923a-c17e0b07f516 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:27.902941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 640/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 10 11:07:27 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:29.924571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fea45-50e6-4f2c-b354-2a81c77628a8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:29.925127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fea45-50e6-4f2c-b354-2a81c77628a8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:29.925127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fea45-50e6-4f2c-b354-2a81c77628a8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:29.926559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 653/1293] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:29.931792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80b6e2-fbac-4bfb-afce-9095ee2a692d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:29.932210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80b6e2-fbac-4bfb-afce-9095ee2a692d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:29.932366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80b6e2-fbac-4bfb-afce-9095ee2a692d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:29.950913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-af80b6e2-fbac-4bfb-afce-9095ee2a692d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:29.952127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-af80b6e2-fbac-4bfb-afce-9095ee2a692d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:29.983227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:30.020525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 641/1294] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:30.043558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fea45-50e6-4f2c-b354-2a81c77628a8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:30.074696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:30.145535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fea45-50e6-4f2c-b354-2a81c77628a8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:30.178994 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6c1b5-197e-4ccd-96b6-c4cb03a156f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:30.179542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6c1b5-197e-4ccd-96b6-c4cb03a156f4 None admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:30.180110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6c1b5-197e-4ccd-96b6-c4cb03a156f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:30.180110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6c1b5-197e-4ccd-96b6-c4cb03a156f4 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:30.180718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6c1b5-197e-4ccd-96b6-c4cb03a156f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:30.182053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6c1b5-197e-4ccd-96b6-c4cb03a156f4 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:30.270322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 654/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Wed Aug 10 11:07:30 2022] POST /identity/v3/services => generated 246 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:07:32.323387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a183-0464-4275-b96f-879456081133 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:32.323948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a183-0464-4275-b96f-879456081133 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:32.323948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08a183-0464-4275-b96f-879456081133 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:32.325396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 642/1296] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:32.330475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:32.330865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:32.331020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:32.348870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:32.350142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:32.422514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 655/1297] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:32.446295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a183-0464-4275-b96f-879456081133 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:32.545103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08a183-0464-4275-b96f-879456081133 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:32.578297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:32.578837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:32.579114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:32.579381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] PATH_INFO: `/v3/services/placement` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:32.579957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:32.581342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:32.590712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:32.591910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:32.593204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 643/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:07:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:32.598015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:32.616337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:32.692855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4487f-7173-4927-9b5d-31add73945e0 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:32.725982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:32.727188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:32.727470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:32.727740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:32.728388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:32.728719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:32.730069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:32.735010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 656/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 10 11:07:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:32.741061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:32.771797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:32.839362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd609dbe-d9c8-4c74-be84-0286c22fc3ad None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:32.874657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:32.875265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:32.875548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:32.875825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:32.876559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] RBAC: Adding query filter params (interface=public, service_id=fc8c900dd8fc477bb48999b2d6c18b61, region_id=RegionOne) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:32.876869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:32.878259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdcb448-d794-45ec-b130-8695f14f049d None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:32.883930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 644/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Wed Aug 10 11:07:32 2022] GET /identity/v3/endpoints?service_id=fc8c900dd8fc477bb48999b2d6c18b61&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:32.889628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:32.971168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:32.988905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3719b593-52e2-493e-b2b3-d2390bfda443 None admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:33.022901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b84ab-4058-443b-8deb-ce047849b049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:33.024585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b84ab-4058-443b-8deb-ce047849b049 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:33.025091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b84ab-4058-443b-8deb-ce047849b049 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:33.025412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b84ab-4058-443b-8deb-ce047849b049 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:33.026355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b84ab-4058-443b-8deb-ce047849b049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:33.027715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b84ab-4058-443b-8deb-ce047849b049 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:33.034309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 657/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Wed Aug 10 11:07:32 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:07:35.039224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811a98-76ba-4846-b026-7dde901bd2ce None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:35.039798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811a98-76ba-4846-b026-7dde901bd2ce None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:35.039798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76811a98-76ba-4846-b026-7dde901bd2ce None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:35.041161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 645/1302] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:07:35.046209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:35.046662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:35.046820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:35.064938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:07:35.066096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:07:35.130756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 658/1303] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:07:35 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 10 11:07:35.154925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76811a98-76ba-4846-b026-7dde901bd2ce None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:35.250867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:35.257529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76811a98-76ba-4846-b026-7dde901bd2ce None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:35.291264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:35.291839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:35.292150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:35.292424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:35.293000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:35.294358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 10 11:07:35.303229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 10 11:07:35.304553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.get_or_create( Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application with Lock( Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self._enter() Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application return self.creator() Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application created_value = creator( Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: ERROR keystone.server.flask.application Aug 10 11:07:35.306894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 646/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Wed Aug 10 11:07:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 10 11:07:35.311296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:35.385598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:35.407884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b05441-aa4a-4edc-af8b-b74e82107721 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:35.441128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:35.441724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:35.442000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:35.442268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] PATH_INFO: `/v3/services` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:35.442885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:07:35.443182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:35.444583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9dc0a-e223-43c8-a99d-24e50a7b9709 None admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:35.449504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 659/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Wed Aug 10 11:07:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:07:35.455444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:07:35.558884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f5da7-d5e1-425c-9b59-b0407f7c3dc2 None admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:07:35.592201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd597-ec15-4f81-a8f4-dff529b13724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:07:35.592799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd597-ec15-4f81-a8f4-dff529b13724 None admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:07:35.593076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd597-ec15-4f81-a8f4-dff529b13724 None admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:07:35.593343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd597-ec15-4f81-a8f4-dff529b13724 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:07:35.593995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdd597-ec15-4f81-a8f4-dff529b13724 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:07:35.595342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdd597-ec15-4f81-a8f4-dff529b13724 None admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:07:35.727630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 647/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:07:35 2022] POST /identity/v3/endpoints => generated 331 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:07:45.526838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=30 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:07:45.722326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:22.744279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ba0eb-3ad1-41dc-aece-d3d8857dc7e5 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:22.746284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ba0eb-3ad1-41dc-aece-d3d8857dc7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:22.747482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462ba0eb-3ad1-41dc-aece-d3d8857dc7e5 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:22.765165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131864-9012-42b6-89b4-fad045041d1e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:22.767222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131864-9012-42b6-89b4-fad045041d1e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:22.767688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131864-9012-42b6-89b4-fad045041d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:22.770293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 660/1307] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:22 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:22.888137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-01131864-9012-42b6-89b4-fad045041d1e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:22.889639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-01131864-9012-42b6-89b4-fad045041d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:23.030225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 648/1308] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:23.086336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd8937-d8bf-4053-ba59-b65761e0e72e None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:23.086640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd8937-d8bf-4053-ba59-b65761e0e72e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:23.086889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd8937-d8bf-4053-ba59-b65761e0e72e None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:23.088601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 661/1309] 10.222.0.13 () {54 vars in 965 bytes} [Wed Aug 10 11:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:23.099630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8928e-847d-4f0e-a423-54d81068c687 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:23.100237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8928e-847d-4f0e-a423-54d81068c687 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:23.104522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b8928e-847d-4f0e-a423-54d81068c687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:23.212248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b3b8928e-847d-4f0e-a423-54d81068c687 None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:23.213598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b3b8928e-847d-4f0e-a423-54d81068c687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:23.240439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:23.307474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 649/1310] 10.222.0.13 () {58 vars in 1062 bytes} [Wed Aug 10 11:14:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:23.317251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd8937-d8bf-4053-ba59-b65761e0e72e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:23.318985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-29bd8937-d8bf-4053-ba59-b65761e0e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:23.439092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd8937-d8bf-4053-ba59-b65761e0e72e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:23.519633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:23.520184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:23.520459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:23.520726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:23.522328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:23.581902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:23.659838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:23.661333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201d8ba-fc99-41e7-a4fe-b411b1bf7fc7 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:23.683665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 662/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:26.090323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0b6ba-6f79-4f9a-80a5-bb25ead2f81b None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:26.090863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0b6ba-6f79-4f9a-80a5-bb25ead2f81b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:26.090863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a0b6ba-6f79-4f9a-80a5-bb25ead2f81b None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:26.092276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 650/1312] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:26.097594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42a0d4-c226-4453-8185-b4be00f9f8ed None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:26.098069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42a0d4-c226-4453-8185-b4be00f9f8ed None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:26.098150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42a0d4-c226-4453-8185-b4be00f9f8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:26.176707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ae42a0d4-c226-4453-8185-b4be00f9f8ed None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:26.177936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ae42a0d4-c226-4453-8185-b4be00f9f8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:26.280367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 663/1313] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:26.334024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52ae0-6356-46c8-9a82-9077ca5895da None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:26.334310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52ae0-6356-46c8-9a82-9077ca5895da None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:26.334582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba52ae0-6356-46c8-9a82-9077ca5895da None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:26.335903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 651/1314] 10.222.0.13 () {54 vars in 965 bytes} [Wed Aug 10 11:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:26.341861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa3ed6-1826-415a-bab3-3821f514b761 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:26.342092 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa3ed6-1826-415a-bab3-3821f514b761 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:26.342364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa3ed6-1826-415a-bab3-3821f514b761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:26.354395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:26.426676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8daa3ed6-1826-415a-bab3-3821f514b761 None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:26.427902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8daa3ed6-1826-415a-bab3-3821f514b761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:26.504927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 664/1315] 10.222.0.13 () {58 vars in 1062 bytes} [Wed Aug 10 11:14:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:26.513778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba52ae0-6356-46c8-9a82-9077ca5895da None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:26.515376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cba52ae0-6356-46c8-9a82-9077ca5895da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:26.620742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba52ae0-6356-46c8-9a82-9077ca5895da None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:26.691034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:26.693863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:26.694457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:26.694731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:26.694997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:26.696615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:26.803819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:26.805346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0bd586-b279-4705-9002-7027005b9dfc service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:26.823541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 652/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:29.553602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4218d-11b4-41de-9ad0-449fda17030e None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:29.554172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4218d-11b4-41de-9ad0-449fda17030e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:29.554172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4218d-11b4-41de-9ad0-449fda17030e None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:29.555453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 665/1317] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:29.560542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c1b62-6dfa-45c4-889a-e3cd8eed3bcc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:29.560965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c1b62-6dfa-45c4-889a-e3cd8eed3bcc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:29.561075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c1b62-6dfa-45c4-889a-e3cd8eed3bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:29.625421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1b8c1b62-6dfa-45c4-889a-e3cd8eed3bcc None None] MFA Rules not processed for user `67dbb8e285ad48eaa50f92f1a4327253`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:29.626542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1b8c1b62-6dfa-45c4-889a-e3cd8eed3bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:29.677415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:29.720775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 653/1318] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:29.760733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4218d-11b4-41de-9ad0-449fda17030e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:29.762266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-71c4218d-11b4-41de-9ad0-449fda17030e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:29.863823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4218d-11b4-41de-9ad0-449fda17030e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:29.866860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:29.936397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:29.936926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:29.937207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:29.937476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:29.939059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:30.043149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:30.044605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a42b4b-7c4f-4c1f-892d-0c85b1ed48ae service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:30.063419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 666/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:32.896422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b77350-1048-4450-bd08-02be843dd682 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:32.896968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b77350-1048-4450-bd08-02be843dd682 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:32.896968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b77350-1048-4450-bd08-02be843dd682 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:32.898295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 654/1320] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:32.903588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d313e-d77c-4775-98e7-cbeb8b2087e2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:32.904031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d313e-d77c-4775-98e7-cbeb8b2087e2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:32.904170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d313e-d77c-4775-98e7-cbeb8b2087e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:32.943792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:32.968454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-866d313e-d77c-4775-98e7-cbeb8b2087e2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:32.970376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-866d313e-d77c-4775-98e7-cbeb8b2087e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:33.065615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 667/1321] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:33.104652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b77350-1048-4450-bd08-02be843dd682 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:33.106172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d4b77350-1048-4450-bd08-02be843dd682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:33.209749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b77350-1048-4450-bd08-02be843dd682 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:33.281996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:33.282599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:33.282876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:33.283147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:33.284790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:33.292009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:33.390453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:33.392698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca8dae-33ab-4b36-8bbb-c8080c69d305 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:33.410351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 655/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:36.418259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3259b9f-f62d-4ccd-b4ac-f2378e41255c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.418787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3259b9f-f62d-4ccd-b4ac-f2378e41255c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.418787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3259b9f-f62d-4ccd-b4ac-f2378e41255c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.420158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 668/1323] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:36.425818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1bcfc-c39c-4b67-ac2b-8acc1074b436 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.426246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1bcfc-c39c-4b67-ac2b-8acc1074b436 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.426363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1bcfc-c39c-4b67-ac2b-8acc1074b436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.491671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a6c1bcfc-c39c-4b67-ac2b-8acc1074b436 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:36.492894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a6c1bcfc-c39c-4b67-ac2b-8acc1074b436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.550436 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:36.589289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 656/1324] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:36.625383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3259b9f-f62d-4ccd-b4ac-f2378e41255c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:36.626962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f3259b9f-f62d-4ccd-b4ac-f2378e41255c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.739702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3259b9f-f62d-4ccd-b4ac-f2378e41255c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:36.753075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:36.816904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:36.817551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:36.817831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:36.818104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:36.819741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:36.928688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:36.930080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c91ded-bba0-468d-a1ec-dbb517ec1736 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:36.948783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 669/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:39.029263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e8026-0e49-4e1c-9044-3c49032f34c7 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.029867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e8026-0e49-4e1c-9044-3c49032f34c7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.029867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e8026-0e49-4e1c-9044-3c49032f34c7 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.031165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 657/1326] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:39.036969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49bb71-6349-4651-955c-276b85c7c289 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.037420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49bb71-6349-4651-955c-276b85c7c289 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.037655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49bb71-6349-4651-955c-276b85c7c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.086369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:39.104588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ea49bb71-6349-4651-955c-276b85c7c289 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:39.105793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ea49bb71-6349-4651-955c-276b85c7c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.203288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 670/1327] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:39.239640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e8026-0e49-4e1c-9044-3c49032f34c7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:39.241220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0b4e8026-0e49-4e1c-9044-3c49032f34c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.346002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e8026-0e49-4e1c-9044-3c49032f34c7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:39.421705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:39.422312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:39.422584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:39.422852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:39.424569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:39.441528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:39.536734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:39.538072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ec0ca-a8fb-4f40-9375-a14ce094084d service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:39.557408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 658/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:41.626652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845caa62-54ed-4ce9-aaa0-d70a327bcfda None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.627167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845caa62-54ed-4ce9-aaa0-d70a327bcfda None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.627167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845caa62-54ed-4ce9-aaa0-d70a327bcfda None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.628479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 671/1329] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:41.633650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3c7c6-2bd7-417c-bb58-97a86b1a47fd None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:41.634111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3c7c6-2bd7-417c-bb58-97a86b1a47fd None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:41.634219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3c7c6-2bd7-417c-bb58-97a86b1a47fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:41.700600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e8b3c7c6-2bd7-417c-bb58-97a86b1a47fd None None] MFA Rules not processed for user `67dbb8e285ad48eaa50f92f1a4327253`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:41.701812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e8b3c7c6-2bd7-417c-bb58-97a86b1a47fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:41.770370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:41.801585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 659/1330] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:14:41.840111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845caa62-54ed-4ce9-aaa0-d70a327bcfda None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:41.841644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-845caa62-54ed-4ce9-aaa0-d70a327bcfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:41.945750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845caa62-54ed-4ce9-aaa0-d70a327bcfda None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:41.966466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:42.022510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:42.023136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:42.023417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:42.023687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:42.025411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:42.134957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:42.136429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2b9b5-0de1-4e0a-ae06-0b7517f7805c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:42.155054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 672/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:41 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:44.660180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830e34d-c1e6-40b3-b41f-4afc999ab259 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:44.660751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830e34d-c1e6-40b3-b41f-4afc999ab259 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:44.660751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830e34d-c1e6-40b3-b41f-4afc999ab259 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:44.662035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 660/1332] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:44.667288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4956d40-1ed5-4890-954f-f1f7faf71192 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:44.667720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4956d40-1ed5-4890-954f-f1f7faf71192 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:44.667905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4956d40-1ed5-4890-954f-f1f7faf71192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:44.733537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:44.736988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e4956d40-1ed5-4890-954f-f1f7faf71192 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:44.738179 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e4956d40-1ed5-4890-954f-f1f7faf71192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:44.830699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 673/1333] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:44.867226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830e34d-c1e6-40b3-b41f-4afc999ab259 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:44.868782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0830e34d-c1e6-40b3-b41f-4afc999ab259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:44.968237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0830e34d-c1e6-40b3-b41f-4afc999ab259 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:45.038588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:45.039179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:45.039454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:45.039720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:45.041308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:45.069396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:45.150802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:45.152194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed895ec-0c2f-406d-b6b1-126c639e5248 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:45.170658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 661/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:47.267513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f7f96-bbf3-4f53-9010-4cb06f2369d2 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:47.268074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f7f96-bbf3-4f53-9010-4cb06f2369d2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:47.268198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f7f96-bbf3-4f53-9010-4cb06f2369d2 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:47.269537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 674/1335] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:47.274753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6308-0eff-4820-96ca-66b0b2365416 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:47.275151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6308-0eff-4820-96ca-66b0b2365416 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:47.275328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f6308-0eff-4820-96ca-66b0b2365416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:47.341511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-bb3f6308-0eff-4820-96ca-66b0b2365416 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:47.342725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bb3f6308-0eff-4820-96ca-66b0b2365416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:47.416248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:47.438243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 662/1336] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:47.479657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f7f96-bbf3-4f53-9010-4cb06f2369d2 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:47.481332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-531f7f96-bbf3-4f53-9010-4cb06f2369d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:47.592295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f7f96-bbf3-4f53-9010-4cb06f2369d2 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:47.620350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:47.665967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:47.666559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:47.666832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:47.667123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:47.668713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:47.775212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:47.776672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c76b46-4375-4435-bac2-698d9d05998b service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:47.794902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 675/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:50.436664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8a524-e02f-430c-a8a0-f2248e0a21b9 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:50.437174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8a524-e02f-430c-a8a0-f2248e0a21b9 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:50.437268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab8a524-e02f-430c-a8a0-f2248e0a21b9 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:50.438536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 663/1338] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:50.444070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8e417-3c9d-4d50-9f10-59814514133d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:50.444496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8e417-3c9d-4d50-9f10-59814514133d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:50.444615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8e417-3c9d-4d50-9f10-59814514133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:50.509081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-90b8e417-3c9d-4d50-9f10-59814514133d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:50.510265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-90b8e417-3c9d-4d50-9f10-59814514133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:50.518899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:50.606683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 676/1339] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:50.643841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8a524-e02f-430c-a8a0-f2248e0a21b9 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:50.645458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5ab8a524-e02f-430c-a8a0-f2248e0a21b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:50.746299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab8a524-e02f-430c-a8a0-f2248e0a21b9 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:50.817322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:50.817915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:50.818187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:50.818448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:50.820072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:50.854052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:50.924567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:50.925916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a027f8b-b086-48fb-8d30-0cdbbe8b141a service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:50.945324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 664/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:55.176933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b415c-8ab6-4bac-a197-0f5fc245e7a8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:55.177495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b415c-8ab6-4bac-a197-0f5fc245e7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:55.177664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b415c-8ab6-4bac-a197-0f5fc245e7a8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:55.179024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 677/1341] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:55.184535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f72f-b0a7-451e-9147-6e03c9ddfb0d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:55.184985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f72f-b0a7-451e-9147-6e03c9ddfb0d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:55.185104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7022f72f-b0a7-451e-9147-6e03c9ddfb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:55.254328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-7022f72f-b0a7-451e-9147-6e03c9ddfb0d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:55.255605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7022f72f-b0a7-451e-9147-6e03c9ddfb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:55.339883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:55.351868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 665/1342] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:55.391520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b415c-8ab6-4bac-a197-0f5fc245e7a8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:55.393267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-130b415c-8ab6-4bac-a197-0f5fc245e7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:55.496280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b415c-8ab6-4bac-a197-0f5fc245e7a8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:55.533375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:55.569688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:55.570357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:55.570840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:55.571372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:55.573389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:55.683366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:55.684867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25568b4-6b3c-44d9-9110-154038345613 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:55.703757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 678/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:14:58.813193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79661f-a531-46eb-8c6f-b418eb859cda None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:58.813700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79661f-a531-46eb-8c6f-b418eb859cda None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:58.813700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f79661f-a531-46eb-8c6f-b418eb859cda None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:58.815009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 666/1344] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:14:58.820539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f067f-4a7f-407d-805e-2dd37229dd56 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:58.820987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f067f-4a7f-407d-805e-2dd37229dd56 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:58.821152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f067f-4a7f-407d-805e-2dd37229dd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:58.886972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-525f067f-4a7f-407d-805e-2dd37229dd56 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:14:58.888123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-525f067f-4a7f-407d-805e-2dd37229dd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:58.906805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:58.987262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 679/1345] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:14:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:14:59.025948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79661f-a531-46eb-8c6f-b418eb859cda None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:14:59.027484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8f79661f-a531-46eb-8c6f-b418eb859cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:59.130013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f79661f-a531-46eb-8c6f-b418eb859cda None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:14:59.203633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:14:59.204894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:14:59.205172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:14:59.205437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:14:59.207016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:14:59.249363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:14:59.322676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:14:59.324140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484c88a-380d-4c4e-8e19-a76581202932 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:14:59.343471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 667/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:14:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:04.254141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6fa33-cf15-4e18-b03f-cd79838cfd68 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:04.254677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6fa33-cf15-4e18-b03f-cd79838cfd68 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:04.254677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6fa33-cf15-4e18-b03f-cd79838cfd68 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:04.256051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 680/1347] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:04.261390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f37e7d-0559-4c55-b289-d7b813ae2734 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:04.261781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f37e7d-0559-4c55-b289-d7b813ae2734 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:04.261937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f37e7d-0559-4c55-b289-d7b813ae2734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:04.333309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d5f37e7d-0559-4c55-b289-d7b813ae2734 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:04.338341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d5f37e7d-0559-4c55-b289-d7b813ae2734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:04.440827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:04.443849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 668/1348] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:04.484771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6fa33-cf15-4e18-b03f-cd79838cfd68 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:04.486914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-afa6fa33-cf15-4e18-b03f-cd79838cfd68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:04.602375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6fa33-cf15-4e18-b03f-cd79838cfd68 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:04.663186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:04.690949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:04.691643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:04.692304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:04.692575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:04.694310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:04.806083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:04.807531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c6994-871f-4509-8e2f-9739c21d1e1c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:04.825805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 681/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:09.709042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14848b-54d7-4095-9c9d-c915cd9a1117 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:09.709581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14848b-54d7-4095-9c9d-c915cd9a1117 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:09.709581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14848b-54d7-4095-9c9d-c915cd9a1117 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:09.711294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 669/1350] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:09.716725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925ed5-38a3-4ba0-90ac-d99dc89a4139 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:09.716725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925ed5-38a3-4ba0-90ac-d99dc89a4139 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:09.717218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3925ed5-38a3-4ba0-90ac-d99dc89a4139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:09.877869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b3925ed5-38a3-4ba0-90ac-d99dc89a4139 None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:09.879339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b3925ed5-38a3-4ba0-90ac-d99dc89a4139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:09.900796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:10.009769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 682/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:15:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:14.149554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a527e4-4932-4349-a373-745e673bbb5a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:14.150083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a527e4-4932-4349-a373-745e673bbb5a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:14.150083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a527e4-4932-4349-a373-745e673bbb5a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:14.151662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 670/1352] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:14.156624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cb0ba-6c20-41ce-99f2-c785f600c73b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:14.157550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cb0ba-6c20-41ce-99f2-c785f600c73b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:14.157781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cb0ba-6c20-41ce-99f2-c785f600c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:14.243216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-380cb0ba-6c20-41ce-99f2-c785f600c73b None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:14.244461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-380cb0ba-6c20-41ce-99f2-c785f600c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:14.320936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:14.359603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 683/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:15:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:15.849424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a7646-5bf5-4ed4-91ba-cdc568d9574f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:15.850003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a7646-5bf5-4ed4-91ba-cdc568d9574f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:15.850096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a7646-5bf5-4ed4-91ba-cdc568d9574f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:15.851506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 671/1354] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:15.856807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3cefb-3157-41a2-b58e-75304d2604be None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:15.857214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3cefb-3157-41a2-b58e-75304d2604be None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:15.857531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3cefb-3157-41a2-b58e-75304d2604be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:15.934277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ced3cefb-3157-41a2-b58e-75304d2604be None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:15.935479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ced3cefb-3157-41a2-b58e-75304d2604be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:16.039944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 684/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:15:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:19.427339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57799968-33c3-4e85-bcf1-03dd90ecb33a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:19.428312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57799968-33c3-4e85-bcf1-03dd90ecb33a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:19.428653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57799968-33c3-4e85-bcf1-03dd90ecb33a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:19.430591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 672/1356] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:19.436771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5b448-2454-4434-bf47-a11fe2dd0d12 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:19.436771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5b448-2454-4434-bf47-a11fe2dd0d12 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:19.437213 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5b448-2454-4434-bf47-a11fe2dd0d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:19.496063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:19.530403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c0f5b448-2454-4434-bf47-a11fe2dd0d12 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:19.532011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c0f5b448-2454-4434-bf47-a11fe2dd0d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:19.637125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 685/1357] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:19.700527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b469b-ddb4-4228-9b36-c190dcbcaf89 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:19.700972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b469b-ddb4-4228-9b36-c190dcbcaf89 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:19.701340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662b469b-ddb4-4228-9b36-c190dcbcaf89 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:19.702771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 673/1358] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 10 11:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:19.711058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef520a-8622-481b-95a9-9cf3eed46b00 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:19.711356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef520a-8622-481b-95a9-9cf3eed46b00 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:19.717030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef520a-8622-481b-95a9-9cf3eed46b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:19.804074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-daef520a-8622-481b-95a9-9cf3eed46b00 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:19.805298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-daef520a-8622-481b-95a9-9cf3eed46b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:19.855797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:19.925220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 686/1359] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 10 11:15:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:19.933004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b469b-ddb4-4228-9b36-c190dcbcaf89 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:19.935578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-662b469b-ddb4-4228-9b36-c190dcbcaf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:20.105923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662b469b-ddb4-4228-9b36-c190dcbcaf89 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:20.183173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:20.225085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:20.225855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:20.226223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:20.226595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:20.228348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:20.324491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:20.326379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ce2ba5-277e-48c8-a335-8294e643c2f7 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:20.346426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 674/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:22.576892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c91b00-8b42-47ed-86b1-d9f74f13e4b4 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:22.577371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c91b00-8b42-47ed-86b1-d9f74f13e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:22.577465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c91b00-8b42-47ed-86b1-d9f74f13e4b4 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:22.579440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 687/1361] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:22.584899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361d4f-d841-4314-8b31-14cb7380ac30 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:22.585296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361d4f-d841-4314-8b31-14cb7380ac30 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:22.585518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74361d4f-d841-4314-8b31-14cb7380ac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:22.657666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-74361d4f-d841-4314-8b31-14cb7380ac30 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:22.658889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-74361d4f-d841-4314-8b31-14cb7380ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:22.698834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:22.762065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 675/1362] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:22.806616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c91b00-8b42-47ed-86b1-d9f74f13e4b4 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:22.808390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-78c91b00-8b42-47ed-86b1-d9f74f13e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:22.987413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c91b00-8b42-47ed-86b1-d9f74f13e4b4 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:22.997430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:23.140021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:23.140642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:23.144911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:23.144911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:23.144911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:23.238704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:23.240341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0997c9-9cba-4d44-8cdd-060813f40eb7 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:23.258169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 688/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:25.795162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d90929-4147-4fa1-99c7-5bf3d4dbf452 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:25.795660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d90929-4147-4fa1-99c7-5bf3d4dbf452 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:25.795660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d90929-4147-4fa1-99c7-5bf3d4dbf452 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:25.797024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 676/1364] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:25.802422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b042-445c-4822-9d62-f4e50f12643c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:25.802843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b042-445c-4822-9d62-f4e50f12643c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:25.803017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b042-445c-4822-9d62-f4e50f12643c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:25.819687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:25.875354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ad41b042-445c-4822-9d62-f4e50f12643c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:25.876670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ad41b042-445c-4822-9d62-f4e50f12643c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:25.980646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 689/1365] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:26.028190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a368c1-df0b-4199-880d-f9117a4b3d56 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:26.028473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a368c1-df0b-4199-880d-f9117a4b3d56 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:26.028927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a368c1-df0b-4199-880d-f9117a4b3d56 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:26.030238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 677/1366] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 10 11:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:26.035751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acafab-2a55-4956-8079-231cc8fea2bf None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:26.036841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acafab-2a55-4956-8079-231cc8fea2bf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:26.036841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78acafab-2a55-4956-8079-231cc8fea2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:26.116631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-78acafab-2a55-4956-8079-231cc8fea2bf None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:26.117915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-78acafab-2a55-4956-8079-231cc8fea2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:26.154752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:26.376872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 690/1367] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 10 11:15:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:26.389203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a368c1-df0b-4199-880d-f9117a4b3d56 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:26.391512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-11a368c1-df0b-4199-880d-f9117a4b3d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:26.548565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a368c1-df0b-4199-880d-f9117a4b3d56 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:26.572183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:26.677271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:26.677793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:26.678065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:26.678329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:26.679955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:26.779632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:26.781252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6172c87-8cf2-4cb3-840a-6097e56edb2e service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:26.801886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 678/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:27.581781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689218f0-51fa-47ea-85fd-608f198019a7 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:27.582308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689218f0-51fa-47ea-85fd-608f198019a7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:27.582472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689218f0-51fa-47ea-85fd-608f198019a7 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:27.583841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 691/1369] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:27.589073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6530-b3ef-4d34-92e5-61f5abeebbde None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:27.589362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6530-b3ef-4d34-92e5-61f5abeebbde None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:27.589640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6530-b3ef-4d34-92e5-61f5abeebbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:27.622615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:27.657844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-848d6530-b3ef-4d34-92e5-61f5abeebbde None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:27.659063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-848d6530-b3ef-4d34-92e5-61f5abeebbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:27.907815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 679/1370] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:28.248241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507f77-721b-4c3f-b777-672a6f9744f3 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.248529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507f77-721b-4c3f-b777-672a6f9744f3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.248805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a507f77-721b-4c3f-b777-672a6f9744f3 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.251534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 692/1371] 10.222.0.13 () {54 vars in 964 bytes} [Wed Aug 10 11:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:28.255813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3574ad-cf3c-4acc-8c46-b48a8812d7bb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.256255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3574ad-cf3c-4acc-8c46-b48a8812d7bb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.256660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3574ad-cf3c-4acc-8c46-b48a8812d7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.336586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9d3574ad-cf3c-4acc-8c46-b48a8812d7bb None None] MFA Rules not processed for user `3cada6c06ef8427395eda888397efac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:28.337800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9d3574ad-cf3c-4acc-8c46-b48a8812d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.398166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:28.430356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 680/1372] 10.222.0.13 () {58 vars in 1061 bytes} [Wed Aug 10 11:15:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:28.437980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507f77-721b-4c3f-b777-672a6f9744f3 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:28.439572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4a507f77-721b-4c3f-b777-672a6f9744f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.551308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a507f77-721b-4c3f-b777-672a6f9744f3 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:28.594704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:28.629331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:28.630059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:28.630206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:28.633000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:28.633000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:28.756620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:28.758432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a5d1c-488d-4e90-a5ca-6c881c58acec service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:28.782819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 693/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:29.057174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9adf-ba38-4125-b126-4038437aaea7 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.057657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9adf-ba38-4125-b126-4038437aaea7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.058054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462b9adf-ba38-4125-b126-4038437aaea7 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.059221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 681/1374] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:29.068643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968f662-732f-4db3-ab1a-f5c6faeacf39 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.068643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968f662-732f-4db3-ab1a-f5c6faeacf39 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.068643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968f662-732f-4db3-ab1a-f5c6faeacf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.145068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f968f662-732f-4db3-ab1a-f5c6faeacf39 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:29.146316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f968f662-732f-4db3-ab1a-f5c6faeacf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:29.169654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:29.246887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 694/1375] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:29.291358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b9adf-ba38-4125-b126-4038437aaea7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:29.292924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-462b9adf-ba38-4125-b126-4038437aaea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:29.449076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462b9adf-ba38-4125-b126-4038437aaea7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:29.501928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:29.564563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:29.565720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:29.565993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:29.566260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:29.568218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:29.657401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:29.658878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa6d7b7-6505-4258-b22b-1ff930266abe service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:29.676874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 682/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:31.837794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5313f1-f228-46cb-9359-84ec9ed4e4ae None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:31.840152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5313f1-f228-46cb-9359-84ec9ed4e4ae None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:31.840152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5313f1-f228-46cb-9359-84ec9ed4e4ae None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:31.840152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 695/1377] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:31.845564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027412-1e0e-4d84-af48-73fd9c0b3109 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:31.845564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027412-1e0e-4d84-af48-73fd9c0b3109 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:31.845564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82027412-1e0e-4d84-af48-73fd9c0b3109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:31.921821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-82027412-1e0e-4d84-af48-73fd9c0b3109 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:31.923675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-82027412-1e0e-4d84-af48-73fd9c0b3109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:31.930609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:32.030903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 683/1378] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:32.077693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5313f1-f228-46cb-9359-84ec9ed4e4ae None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:32.079504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9a5313f1-f228-46cb-9359-84ec9ed4e4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:32.231652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5313f1-f228-46cb-9359-84ec9ed4e4ae None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:32.236283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:32.360481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:32.361934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:32.361934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:32.361934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:32.364154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:32.501487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:32.501487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db35162-e0ee-4832-aedb-704d0fa47063 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:32.521694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 696/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:34.716188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f287ad-4e06-4f44-8ecc-799f640b3da6 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:34.716663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f287ad-4e06-4f44-8ecc-799f640b3da6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:34.716741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f287ad-4e06-4f44-8ecc-799f640b3da6 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:34.718240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 684/1380] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:34.723295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f772-428a-4ae6-b21b-5b1fa1fa528b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:34.723762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f772-428a-4ae6-b21b-5b1fa1fa528b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:34.723937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc6f772-428a-4ae6-b21b-5b1fa1fa528b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:34.735969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:34.793276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ebc6f772-428a-4ae6-b21b-5b1fa1fa528b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:34.794486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ebc6f772-428a-4ae6-b21b-5b1fa1fa528b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:34.904030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 697/1381] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:34.944647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f287ad-4e06-4f44-8ecc-799f640b3da6 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:34.946107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-80f287ad-4e06-4f44-8ecc-799f640b3da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:35.091938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:35.103384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f287ad-4e06-4f44-8ecc-799f640b3da6 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:35.223320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:35.223846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:35.224161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:35.224432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:35.226170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:35.333161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:35.334785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf438467-ad7d-4a7c-9032-8a45705b648e service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:35.346127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:35.354946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 685/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:37.558703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7419-2235-4896-89e4-927e19c2942d None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:37.559266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7419-2235-4896-89e4-927e19c2942d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:37.559266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c7419-2235-4896-89e4-927e19c2942d None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:37.560614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 698/1383] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:37.567320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93317-c25c-4585-9f40-4439f0c1f31c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:37.567749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93317-c25c-4585-9f40-4439f0c1f31c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:37.568031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93317-c25c-4585-9f40-4439f0c1f31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:37.638176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-16a93317-c25c-4585-9f40-4439f0c1f31c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:37.639336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-16a93317-c25c-4585-9f40-4439f0c1f31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:37.740028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 686/1384] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:37.783355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7419-2235-4896-89e4-927e19c2942d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:37.784920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d68c7419-2235-4896-89e4-927e19c2942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:37.872679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:37.943161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c7419-2235-4896-89e4-927e19c2942d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:38.073356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:38.074133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:38.074762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:38.075140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:38.077199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:38.133103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:38.173844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:38.175444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdaef85-8dbd-4732-b6cd-a88e579b96e1 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:38.194363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 699/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:40.518245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd1721-4f15-41f9-8e52-cdb9908653f5 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:40.518828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd1721-4f15-41f9-8e52-cdb9908653f5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:40.518828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd1721-4f15-41f9-8e52-cdb9908653f5 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:40.520202 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 687/1386] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:40.525559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6f58-066c-4d19-9a70-d5d32980e003 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:40.526053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6f58-066c-4d19-9a70-d5d32980e003 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:40.526220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f6f58-066c-4d19-9a70-d5d32980e003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:40.600228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-895f6f58-066c-4d19-9a70-d5d32980e003 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:40.601445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-895f6f58-066c-4d19-9a70-d5d32980e003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:40.705967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 700/1387] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:40.750855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd1721-4f15-41f9-8e52-cdb9908653f5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:40.753030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-72dd1721-4f15-41f9-8e52-cdb9908653f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:40.839680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:40.917727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd1721-4f15-41f9-8e52-cdb9908653f5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:41.043370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:41.043897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:41.044214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:41.044488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:41.046117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:41.086266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:41.138889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:41.140510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2e92e-cbcd-4e55-a5ed-3fc13ddc8a80 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:41.159518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 688/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:43.319390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08d477-bdd5-44f4-a3f6-449d7571fbbc None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:43.320019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08d477-bdd5-44f4-a3f6-449d7571fbbc None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:43.320150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08d477-bdd5-44f4-a3f6-449d7571fbbc None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:43.321377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 701/1389] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:43.326534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a45c02-7998-4216-ac95-0fb8f078f0b6 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:43.326981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a45c02-7998-4216-ac95-0fb8f078f0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:43.327159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a45c02-7998-4216-ac95-0fb8f078f0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:43.395669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-16a45c02-7998-4216-ac95-0fb8f078f0b6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:43.396937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-16a45c02-7998-4216-ac95-0fb8f078f0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:43.493932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 689/1390] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:43.538855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08d477-bdd5-44f4-a3f6-449d7571fbbc None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:43.540435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ec08d477-bdd5-44f4-a3f6-449d7571fbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:43.571909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:43.685967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08d477-bdd5-44f4-a3f6-449d7571fbbc None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:43.785446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:43.796557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:43.797638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:43.797638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:43.798344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:43.799871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:43.889178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:43.890692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd622036-dbd4-4fe1-803f-123b1898f302 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:43.909716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 702/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:44.323937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3dee2-90bb-4a3e-be55-0ef477df3305 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:44.324257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3dee2-90bb-4a3e-be55-0ef477df3305 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:44.324521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3dee2-90bb-4a3e-be55-0ef477df3305 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:44.325888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 690/1392] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:44.332482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7831d-f96e-49fc-9a9c-587b8561f9cf None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:44.332959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7831d-f96e-49fc-9a9c-587b8561f9cf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:44.333071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb7831d-f96e-49fc-9a9c-587b8561f9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:44.400305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2cb7831d-f96e-49fc-9a9c-587b8561f9cf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:44.401517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2cb7831d-f96e-49fc-9a9c-587b8561f9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:44.460412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:44.502842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 703/1393] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:44.557809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1603b6-52cd-405f-aa18-abd233584b0e None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:44.558289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1603b6-52cd-405f-aa18-abd233584b0e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:44.558434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1603b6-52cd-405f-aa18-abd233584b0e None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:44.559783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 691/1394] 10.222.0.13 () {54 vars in 967 bytes} [Wed Aug 10 11:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:44.567371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb5da-cf07-4bd9-991d-aa9c6bf475ca None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:44.567686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb5da-cf07-4bd9-991d-aa9c6bf475ca None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:44.567961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3cb5da-cf07-4bd9-991d-aa9c6bf475ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:44.645603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ed3cb5da-cf07-4bd9-991d-aa9c6bf475ca None None] MFA Rules not processed for user `3a95ab030c9940c9946c64c82567773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:44.646705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ed3cb5da-cf07-4bd9-991d-aa9c6bf475ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:44.724404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 704/1395] 10.222.0.13 () {58 vars in 1064 bytes} [Wed Aug 10 11:15:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:44.731843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1603b6-52cd-405f-aa18-abd233584b0e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:44.733378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4c1603b6-52cd-405f-aa18-abd233584b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:44.748136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:44.839269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1603b6-52cd-405f-aa18-abd233584b0e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:44.916053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:44.916593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:44.916868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:44.917138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:44.918743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:44.989244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:45.030099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:45.031579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ca7c0-0b64-44a3-a6f7-17610e3e0c3e service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:45.051755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 692/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:45.968955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c4fda-1cda-48be-a22b-2bf7eaceb653 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:45.969455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c4fda-1cda-48be-a22b-2bf7eaceb653 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:45.969562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121c4fda-1cda-48be-a22b-2bf7eaceb653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.037333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-121c4fda-1cda-48be-a22b-2bf7eaceb653 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:46.038529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-121c4fda-1cda-48be-a22b-2bf7eaceb653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.050400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:46.066818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481b93f-b307-47bf-88f5-6fa815e83c1a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.067619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481b93f-b307-47bf-88f5-6fa815e83c1a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.067619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481b93f-b307-47bf-88f5-6fa815e83c1a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.070945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 693/1397] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:46.074165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.074453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.074731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.146009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:46.147236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.152180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 705/1398] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:15:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:46.200607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d14d78-538e-4d59-9f7b-8b94b8fc4bfa None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.200879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d14d78-538e-4d59-9f7b-8b94b8fc4bfa None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.201432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d14d78-538e-4d59-9f7b-8b94b8fc4bfa None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.203013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 706/1399] 10.222.0.13 () {54 vars in 943 bytes} [Wed Aug 10 11:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:46.211128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.211569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.211744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.257092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 694/1400] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:15:46.300974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] MFA Rules not processed for user `dfb941d3781044b0a2092c596bf51e32`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:46.302482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.303615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:46.304982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.336376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:46.395922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:46.419462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 707/1401] 10.222.0.13 () {58 vars in 1040 bytes} [Wed Aug 10 11:15:46 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:46.436090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:46.444456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d88d82-f266-46b1-9a68-983fad768e66 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:46.447230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883f4c-3dc6-4929-99d3-c7d5a6086e2c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:46.552943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:46.553467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.553743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.554013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.555658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.556873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:46.557483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:46.557760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:46.558029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:46.559601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:46.579401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:46.642090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:46.643750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40d19b-1b9f-47d5-9dd4-b8b669ec57cf service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:46.651638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 708/1402] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:15:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:15:46.651789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:46.653488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a65b76-7e33-47c9-bf10-fa3daa2bedcb service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:46.676066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 695/1403] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:49.021577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a943-6162-4e59-83f6-def136df6b92 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:49.022106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a943-6162-4e59-83f6-def136df6b92 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:49.022106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d0a943-6162-4e59-83f6-def136df6b92 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:49.023461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 709/1404] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:15:49.029008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655977b-109f-4480-b391-dae08cd279eb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:49.029495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655977b-109f-4480-b391-dae08cd279eb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:49.029608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655977b-109f-4480-b391-dae08cd279eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:49.102833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1655977b-109f-4480-b391-dae08cd279eb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:49.104088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1655977b-109f-4480-b391-dae08cd279eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:49.210502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:49.217499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 696/1405] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:49.263804 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a943-6162-4e59-83f6-def136df6b92 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:49.265278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-72d0a943-6162-4e59-83f6-def136df6b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:49.275782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:49.467766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d0a943-6162-4e59-83f6-def136df6b92 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:49.576299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:49.614399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:49.620717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:49.620717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:49.620717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:49.620717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:49.721552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:49.723433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c81cd1-310f-41c1-9f5a-6eacbfd3f76f service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:49.751678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 710/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:15:52.285324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2fdee-ce88-4397-aa08-c4454fb2a614 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:52.285900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2fdee-ce88-4397-aa08-c4454fb2a614 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:52.285900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f2fdee-ce88-4397-aa08-c4454fb2a614 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:52.287197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 697/1407] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:15:52.292422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea9c68-9cec-4ba0-b708-cad92369e613 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:52.293216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea9c68-9cec-4ba0-b708-cad92369e613 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:52.293492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea9c68-9cec-4ba0-b708-cad92369e613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:52.365459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-13ea9c68-9cec-4ba0-b708-cad92369e613 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:15:52.366936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-13ea9c68-9cec-4ba0-b708-cad92369e613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:52.425852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:52.481028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 711/1408] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:15:52.530230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2fdee-ce88-4397-aa08-c4454fb2a614 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:15:52.530230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d1f2fdee-ce88-4397-aa08-c4454fb2a614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:52.697774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f2fdee-ce88-4397-aa08-c4454fb2a614 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:15:52.775814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:15:52.837225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:15:52.838065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:15:52.838341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:15:52.838603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:15:52.840227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:15:52.941889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:15:52.944311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc33b51-bd41-4d48-acdb-86263627c613 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:15:52.964389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 698/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:02.478770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=23 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:02.962425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=64 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:10.681645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f9ff3-312c-445c-8b79-8d60cee522f1 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:10.682190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f9ff3-312c-445c-8b79-8d60cee522f1 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:10.682635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f9ff3-312c-445c-8b79-8d60cee522f1 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:10.684156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 712/1410] 10.222.0.13 () {54 vars in 918 bytes} [Wed Aug 10 11:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:10.689915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308f9c7-13e2-4de2-812b-b0d27f22afb6 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:10.690372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308f9c7-13e2-4de2-812b-b0d27f22afb6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:10.690491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308f9c7-13e2-4de2-812b-b0d27f22afb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:10.776732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0308f9c7-13e2-4de2-812b-b0d27f22afb6 None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:10.777952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0308f9c7-13e2-4de2-812b-b0d27f22afb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:10.883178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 699/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 10 11:16:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:10.898020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af99b23-3afd-4ceb-a6ec-2084ccd0a183 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:10.898306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af99b23-3afd-4ceb-a6ec-2084ccd0a183 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:10.898578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af99b23-3afd-4ceb-a6ec-2084ccd0a183 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:10.899882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 713/1412] 10.222.0.13 () {54 vars in 918 bytes} [Wed Aug 10 11:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:10.904402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02acfeb-3b5e-4efb-a9a0-e3773162da76 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:10.904684 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02acfeb-3b5e-4efb-a9a0-e3773162da76 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:10.904951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02acfeb-3b5e-4efb-a9a0-e3773162da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:10.982501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:10.985239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e02acfeb-3b5e-4efb-a9a0-e3773162da76 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:10.986703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e02acfeb-3b5e-4efb-a9a0-e3773162da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:11.108855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 700/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Wed Aug 10 11:16:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:13.611931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121d92-ab4e-4b6c-8a49-507287df7f0c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:13.612508 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121d92-ab4e-4b6c-8a49-507287df7f0c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:13.612508 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac121d92-ab4e-4b6c-8a49-507287df7f0c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:13.613823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:13.619016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53322f-cc2d-4ecb-b1c3-72e2ebc8a255 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:13.619434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53322f-cc2d-4ecb-b1c3-72e2ebc8a255 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:13.619562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53322f-cc2d-4ecb-b1c3-72e2ebc8a255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:13.684736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0a53322f-cc2d-4ecb-b1c3-72e2ebc8a255 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:13.685932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0a53322f-cc2d-4ecb-b1c3-72e2ebc8a255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:13.739888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:13.781035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:13.824448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac121d92-ab4e-4b6c-8a49-507287df7f0c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:13.825992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ac121d92-ab4e-4b6c-8a49-507287df7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:13.968998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac121d92-ab4e-4b6c-8a49-507287df7f0c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:14.039754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:14.078768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:14.079371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:14.079643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:14.079906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:14.081605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:14.170400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:14.172053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f81a0-022b-42e3-b963-05de6642daa1 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:14.190322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 715/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:15.096892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1fc50-41f7-48ee-82ae-3df5ef55a551 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:15.097513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1fc50-41f7-48ee-82ae-3df5ef55a551 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:15.097513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1fc50-41f7-48ee-82ae-3df5ef55a551 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:15.098757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:15.103740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3983cb-e571-4f59-ab73-79af70b832cf None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:15.104185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3983cb-e571-4f59-ab73-79af70b832cf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:15.104337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3983cb-e571-4f59-ab73-79af70b832cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:15.178822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fc3983cb-e571-4f59-ab73-79af70b832cf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:15.180040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fc3983cb-e571-4f59-ab73-79af70b832cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:15.206668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:15.275066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:15.427461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1fc50-41f7-48ee-82ae-3df5ef55a551 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:15.429042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-15e1fc50-41f7-48ee-82ae-3df5ef55a551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:15.573406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1fc50-41f7-48ee-82ae-3df5ef55a551 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:15.613443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:15.685880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:15.686414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:15.686690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:15.686958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:15.689517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:15.778185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:15.779772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899763a-14ff-42c3-93c5-85272aee8fe2 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:15.798128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 703/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:16.675110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04db599-e77d-43f4-9757-86afed64cc46 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:16.675635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04db599-e77d-43f4-9757-86afed64cc46 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:16.675635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04db599-e77d-43f4-9757-86afed64cc46 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:16.676923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:16.681897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280602ef-373f-4e2f-9d3c-af82a0a77058 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:16.682344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280602ef-373f-4e2f-9d3c-af82a0a77058 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:16.682444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280602ef-373f-4e2f-9d3c-af82a0a77058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:16.736626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:16.757430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-280602ef-373f-4e2f-9d3c-af82a0a77058 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:16.758634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-280602ef-373f-4e2f-9d3c-af82a0a77058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:16.854948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:17.008612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04db599-e77d-43f4-9757-86afed64cc46 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:17.010117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b04db599-e77d-43f4-9757-86afed64cc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:17.160415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04db599-e77d-43f4-9757-86afed64cc46 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:17.172340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:17.271220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:17.271739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:17.272039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:17.272294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:17.273877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:17.361301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:17.362919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b786bf2-f89b-4742-b8a4-8d5980a1c9ba service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:17.380741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 718/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:18.244320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fece1f-41ad-437f-ae34-d59dfdc4e93a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:18.244894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fece1f-41ad-437f-ae34-d59dfdc4e93a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:18.244894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fece1f-41ad-437f-ae34-d59dfdc4e93a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:18.246211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:18.251131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a0e6-e3ed-4402-9529-a323853d82dd None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:18.251585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a0e6-e3ed-4402-9529-a323853d82dd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:18.251708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59a0e6-e3ed-4402-9529-a323853d82dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:18.271096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:18.325037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-be59a0e6-e3ed-4402-9529-a323853d82dd None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:18.326222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-be59a0e6-e3ed-4402-9529-a323853d82dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:18.428807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:18.588041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fece1f-41ad-437f-ae34-d59dfdc4e93a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:18.589584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-19fece1f-41ad-437f-ae34-d59dfdc4e93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:18.699475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:18.737437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fece1f-41ad-437f-ae34-d59dfdc4e93a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:18.848751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:18.849277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:18.849553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:18.849814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:18.851408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:18.925220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:18.938085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:18.939695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122ccd1-3e82-40d3-8ab5-e1878016a0b3 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:18.958258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 706/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:19.824962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11402d-1ce6-4bca-952e-894818a9fc8e None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:19.825509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11402d-1ce6-4bca-952e-894818a9fc8e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:19.825623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11402d-1ce6-4bca-952e-894818a9fc8e None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:19.826907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:19.831816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca773a4-df79-49b2-b52b-d0fa87c8851d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:19.832263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca773a4-df79-49b2-b52b-d0fa87c8851d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:19.832400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca773a4-df79-49b2-b52b-d0fa87c8851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:19.906464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cca773a4-df79-49b2-b52b-d0fa87c8851d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:19.907793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cca773a4-df79-49b2-b52b-d0fa87c8851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:20.003833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:20.162608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11402d-1ce6-4bca-952e-894818a9fc8e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:20.164251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-dc11402d-1ce6-4bca-952e-894818a9fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:20.247002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:20.308696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11402d-1ce6-4bca-952e-894818a9fc8e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:20.421375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:20.421975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:20.422248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:20.422512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:20.424125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:20.472468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:20.512747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:20.514334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3f221-6985-472d-8b97-270c5d4c52f1 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:20.532839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 721/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:21.390484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbbd3-56c5-4181-9d72-14765bca1d1f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:21.391276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbbd3-56c5-4181-9d72-14765bca1d1f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:21.391276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3abbbd3-56c5-4181-9d72-14765bca1d1f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:21.392462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:21.398474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7edc9d-7b11-40a8-b36a-800fcbdd3661 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:21.398903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7edc9d-7b11-40a8-b36a-800fcbdd3661 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:21.399005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7edc9d-7b11-40a8-b36a-800fcbdd3661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:21.478782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-cc7edc9d-7b11-40a8-b36a-800fcbdd3661 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:21.479990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-cc7edc9d-7b11-40a8-b36a-800fcbdd3661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:21.578241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:21.730307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbbd3-56c5-4181-9d72-14765bca1d1f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:21.731852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a3abbbd3-56c5-4181-9d72-14765bca1d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:21.780399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:21.878666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3abbbd3-56c5-4181-9d72-14765bca1d1f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:21.992426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:21.992955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:21.993234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:21.993504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:21.995112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:22.001993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:22.084937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:22.086509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f4e72-f42f-4f64-bee1-26924e0cced9 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:22.104439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 709/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:23.019427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a85ee-4074-4968-97e5-65382ddadb6e None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:23.020045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a85ee-4074-4968-97e5-65382ddadb6e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:23.020159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877a85ee-4074-4968-97e5-65382ddadb6e None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:23.022098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:23.027044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16876-c965-4edc-8da7-9e045602695b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:23.027459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16876-c965-4edc-8da7-9e045602695b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:23.027621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b16876-c965-4edc-8da7-9e045602695b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:23.106146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e9b16876-c965-4edc-8da7-9e045602695b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:23.107349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e9b16876-c965-4edc-8da7-9e045602695b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:23.176169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:23.205585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:23.357730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a85ee-4074-4968-97e5-65382ddadb6e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:23.359245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-877a85ee-4074-4968-97e5-65382ddadb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:23.377540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:23.507411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877a85ee-4074-4968-97e5-65382ddadb6e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:23.593587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:23.618680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:23.619265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:23.619543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:23.619815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:23.621425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:23.709127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:23.710711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c4cb3-9fb6-4294-8e13-fcd9c651c899 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:23.728356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 724/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:24.582253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acaf48-0562-4abd-ac64-d5fc36d26929 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:24.582809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acaf48-0562-4abd-ac64-d5fc36d26929 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:24.582809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74acaf48-0562-4abd-ac64-d5fc36d26929 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:24.584363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:24.589168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90122d8-9129-451d-9640-050754b24586 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:24.589565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90122d8-9129-451d-9640-050754b24586 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:24.589712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90122d8-9129-451d-9640-050754b24586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:24.667026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c90122d8-9129-451d-9640-050754b24586 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:24.668215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c90122d8-9129-451d-9640-050754b24586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:24.709138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:24.762801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:16:24.914036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acaf48-0562-4abd-ac64-d5fc36d26929 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:24.915576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-74acaf48-0562-4abd-ac64-d5fc36d26929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:25.061172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74acaf48-0562-4abd-ac64-d5fc36d26929 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:25.115564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:25.174200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:25.175640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:25.175640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:25.175914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:25.177554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:25.269395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:25.271090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5322c36d-4f50-4480-8459-5a3621e430a3 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:25.289488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 712/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:26.213876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a49a0-02b2-45ef-a526-d3f6cab0e8c7 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:26.214456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a49a0-02b2-45ef-a526-d3f6cab0e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:26.214687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a49a0-02b2-45ef-a526-d3f6cab0e8c7 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:26.216378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:26.221332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a3d1e-8aa4-4bfb-92f6-b171850038fc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:26.222310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a3d1e-8aa4-4bfb-92f6-b171850038fc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:26.222534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a3d1e-8aa4-4bfb-92f6-b171850038fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:26.298667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-059a3d1e-8aa4-4bfb-92f6-b171850038fc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:26.300516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-059a3d1e-8aa4-4bfb-92f6-b171850038fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:26.302515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:26.397810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:26.551449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a49a0-02b2-45ef-a526-d3f6cab0e8c7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:26.552899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ee2a49a0-02b2-45ef-a526-d3f6cab0e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:26.698151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a49a0-02b2-45ef-a526-d3f6cab0e8c7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:26.725403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:26.809617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:26.810144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:26.810417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:26.810684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:26.812289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:26.900758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:26.902362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be292e-2e9f-4034-b193-d82ac9d20516 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:26.919925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 727/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:28.840981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6cab8-46a5-4f5c-97d9-38bf1a9c6ffa None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:28.841517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6cab8-46a5-4f5c-97d9-38bf1a9c6ffa None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:28.841517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c6cab8-46a5-4f5c-97d9-38bf1a9c6ffa None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:28.842828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:28.848781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37875-5e47-4d85-8152-985ba9f4c9ae None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:28.849215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37875-5e47-4d85-8152-985ba9f4c9ae None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:28.849327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec37875-5e47-4d85-8152-985ba9f4c9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:28.889254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:28.915601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6ec37875-5e47-4d85-8152-985ba9f4c9ae None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:28.916925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6ec37875-5e47-4d85-8152-985ba9f4c9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:29.012029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:29.051324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6cab8-46a5-4f5c-97d9-38bf1a9c6ffa None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:29.052933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-84c6cab8-46a5-4f5c-97d9-38bf1a9c6ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:29.159961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c6cab8-46a5-4f5c-97d9-38bf1a9c6ffa None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:29.191689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:29.232342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:29.232935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:29.233208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:29.233474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:29.235046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:29.340675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:29.342090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c7cafb-9e02-427b-8d6d-9f8aaa679b88 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:29.359825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 715/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:31.538777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791ade6-8ef8-4b73-9e25-4603e7eefeba None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:31.539360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791ade6-8ef8-4b73-9e25-4603e7eefeba None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:31.539360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791ade6-8ef8-4b73-9e25-4603e7eefeba None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:31.540689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:31.545872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2561d6-9723-4150-bcef-378f359a5999 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:31.546366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2561d6-9723-4150-bcef-378f359a5999 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:31.546493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2561d6-9723-4150-bcef-378f359a5999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:31.615157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-6e2561d6-9723-4150-bcef-378f359a5999 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:31.616486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6e2561d6-9723-4150-bcef-378f359a5999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:31.630134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:31.715267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:31.755417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791ade6-8ef8-4b73-9e25-4603e7eefeba None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:31.757046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2791ade6-8ef8-4b73-9e25-4603e7eefeba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:31.858848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791ade6-8ef8-4b73-9e25-4603e7eefeba None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:31.871235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:31.934859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:31.935416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:31.935690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:31.935956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:31.937551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:32.044792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:32.046206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61677f-72ea-47a1-88c7-9f888d046d1c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:32.066435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 730/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:34.548893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f4a2d-af2d-4477-9238-873cd7185944 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:34.549441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f4a2d-af2d-4477-9238-873cd7185944 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:34.549441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f4a2d-af2d-4477-9238-873cd7185944 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:34.551331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:34.556870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8773d0-0bf1-4c92-a6cb-cdc511bba5f7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:34.557288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8773d0-0bf1-4c92-a6cb-cdc511bba5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:34.557410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8773d0-0bf1-4c92-a6cb-cdc511bba5f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:34.605813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:34.623680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5d8773d0-0bf1-4c92-a6cb-cdc511bba5f7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:34.624845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5d8773d0-0bf1-4c92-a6cb-cdc511bba5f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:34.721243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:34.761690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f4a2d-af2d-4477-9238-873cd7185944 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:34.763216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-015f4a2d-af2d-4477-9238-873cd7185944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:34.864641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f4a2d-af2d-4477-9238-873cd7185944 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:34.904395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:34.939496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:34.940174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:34.940467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:34.940729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:34.942305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:35.047610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:35.049065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2033b9-e5b5-4059-acb1-92f9fc6a7a0c service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:35.067977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 718/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:37.217657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34dcaa-ad0c-4502-9630-0894dc6b0d09 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:37.218227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34dcaa-ad0c-4502-9630-0894dc6b0d09 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:37.218227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34dcaa-ad0c-4502-9630-0894dc6b0d09 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:37.219799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:37.224987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078d095-ad25-40be-b5f0-6613e01700a3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:37.225411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078d095-ad25-40be-b5f0-6613e01700a3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:37.225540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078d095-ad25-40be-b5f0-6613e01700a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:37.291526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5078d095-ad25-40be-b5f0-6613e01700a3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:37.292757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5078d095-ad25-40be-b5f0-6613e01700a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:37.313359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:37.391840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:37.431817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34dcaa-ad0c-4502-9630-0894dc6b0d09 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:37.433380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2b34dcaa-ad0c-4502-9630-0894dc6b0d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:37.534389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34dcaa-ad0c-4502-9630-0894dc6b0d09 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:37.553721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:37.607054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:37.607692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:37.608029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:37.608414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:37.610050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:37.715343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:37.716800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6497dec-d283-4495-91df-1a0df7592c57 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:37.736342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 733/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:39.853814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ecf1-7362-48e3-8b6b-eff6b77485f8 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:39.854382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ecf1-7362-48e3-8b6b-eff6b77485f8 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:39.854382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346ecf1-7362-48e3-8b6b-eff6b77485f8 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:39.855666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:39.860799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfeceb-e74a-4f5f-8288-7c8988dd5fc5 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:39.861193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfeceb-e74a-4f5f-8288-7c8988dd5fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:39.861332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfeceb-e74a-4f5f-8288-7c8988dd5fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:39.922316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:39.925245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d1dfeceb-e74a-4f5f-8288-7c8988dd5fc5 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:39.926434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d1dfeceb-e74a-4f5f-8288-7c8988dd5fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:40.021032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:40.059626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346ecf1-7362-48e3-8b6b-eff6b77485f8 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:40.061218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9346ecf1-7362-48e3-8b6b-eff6b77485f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:40.168320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346ecf1-7362-48e3-8b6b-eff6b77485f8 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:40.215566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:40.239701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:40.240329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:40.240603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:40.240868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:40.242568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:40.349760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:40.351577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c474c6-5ff8-4aaf-8ad4-97713e913ca9 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:40.370458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 721/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:42.428195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5964b2-202a-4168-a611-6d0a52c4bfdf None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:42.428743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5964b2-202a-4168-a611-6d0a52c4bfdf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:42.428743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5964b2-202a-4168-a611-6d0a52c4bfdf None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:42.430063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:42.435449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42a6f-7481-4481-8477-e7e8f62f77b9 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:42.435873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42a6f-7481-4481-8477-e7e8f62f77b9 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:42.436020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa42a6f-7481-4481-8477-e7e8f62f77b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:42.502617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5fa42a6f-7481-4481-8477-e7e8f62f77b9 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:42.503964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5fa42a6f-7481-4481-8477-e7e8f62f77b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:42.532131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:42.599601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:42.637824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5964b2-202a-4168-a611-6d0a52c4bfdf None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:42.639370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-eb5964b2-202a-4168-a611-6d0a52c4bfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:42.740324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5964b2-202a-4168-a611-6d0a52c4bfdf None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:42.767928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:42.814140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:42.814737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:42.815013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:42.815514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:42.817030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:42.921627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:42.922959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599816ef-3136-4022-981c-9de2316ab8ca service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:42.941808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 736/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:44.991593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf62152-95c3-4e25-bf9b-6bb677bd04b1 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:44.992150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf62152-95c3-4e25-bf9b-6bb677bd04b1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:44.992150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf62152-95c3-4e25-bf9b-6bb677bd04b1 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:44.993567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:44.998711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b089-da6b-4979-9bc7-1e2c62b33b24 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:44.999150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b089-da6b-4979-9bc7-1e2c62b33b24 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:44.999266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b85b089-da6b-4979-9bc7-1e2c62b33b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:45.062725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9b85b089-da6b-4979-9bc7-1e2c62b33b24 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:45.063912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9b85b089-da6b-4979-9bc7-1e2c62b33b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:45.072432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:45.160501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:45.197580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf62152-95c3-4e25-bf9b-6bb677bd04b1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:45.199396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-dbf62152-95c3-4e25-bf9b-6bb677bd04b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:45.302727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf62152-95c3-4e25-bf9b-6bb677bd04b1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:45.357137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:45.375546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:45.376186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:45.376465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:45.376733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:45.378359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:45.488858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:45.490433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932b91-0fcc-4aed-a036-6df2322347e3 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:45.509657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 724/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:47.557397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbea99d-7b71-4575-ae55-d993b09bf4ae None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:47.557956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbea99d-7b71-4575-ae55-d993b09bf4ae None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:47.557956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbea99d-7b71-4575-ae55-d993b09bf4ae None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:47.559267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:47.564422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b899e0-9625-4f27-b36e-8e99b89f2451 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:47.564911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b899e0-9625-4f27-b36e-8e99b89f2451 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:47.565127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b899e0-9625-4f27-b36e-8e99b89f2451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:47.629743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d8b899e0-9625-4f27-b36e-8e99b89f2451 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:47.630864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d8b899e0-9625-4f27-b36e-8e99b89f2451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:47.668474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:47.725671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:47.762880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbea99d-7b71-4575-ae55-d993b09bf4ae None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:47.764442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5cbea99d-7b71-4575-ae55-d993b09bf4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:47.873381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbea99d-7b71-4575-ae55-d993b09bf4ae None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:47.918379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:47.954374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:47.954993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:47.955266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:47.956201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:47.957793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:48.062285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:48.063706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d3b3a5-457e-48c2-9159-7ab246952be5 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:48.082494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 739/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:50.108514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf1e4-f7a2-443e-b8c4-6be3f06cb228 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:50.109046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf1e4-f7a2-443e-b8c4-6be3f06cb228 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:50.109180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdf1e4-f7a2-443e-b8c4-6be3f06cb228 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:50.110561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:50.116631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962abbc-c625-4e4b-bec1-718a470f9ede None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:50.117054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962abbc-c625-4e4b-bec1-718a470f9ede None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:50.117165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962abbc-c625-4e4b-bec1-718a470f9ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:50.184057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9962abbc-c625-4e4b-bec1-718a470f9ede None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:50.185252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9962abbc-c625-4e4b-bec1-718a470f9ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:50.203226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:50.280573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:50.318616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf1e4-f7a2-443e-b8c4-6be3f06cb228 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:50.320182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-72cdf1e4-f7a2-443e-b8c4-6be3f06cb228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:50.424733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdf1e4-f7a2-443e-b8c4-6be3f06cb228 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:50.495948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:50.502789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:50.503403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:50.503681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:50.504045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:50.506748 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:50.615108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:50.616590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867e710-9d80-44a2-bbe5-ed74a8ad6534 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:50.634354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 727/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:16:52.658584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8fb1-ffdb-4177-a619-f5e99fc97632 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:52.659150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8fb1-ffdb-4177-a619-f5e99fc97632 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:52.659254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8fb1-ffdb-4177-a619-f5e99fc97632 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:52.660642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:16:52.665978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a4d1-4809-4235-aa7a-35457051516d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:52.666448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a4d1-4809-4235-aa7a-35457051516d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:52.666558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a4d1-4809-4235-aa7a-35457051516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:52.733157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0cc2a4d1-4809-4235-aa7a-35457051516d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:16:52.734393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0cc2a4d1-4809-4235-aa7a-35457051516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:52.782360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:52.831605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:16:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:16:52.868918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8fb1-ffdb-4177-a619-f5e99fc97632 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:16:52.870455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-779b8fb1-ffdb-4177-a619-f5e99fc97632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:52.974036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b8fb1-ffdb-4177-a619-f5e99fc97632 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:16:53.020260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:16:53.046681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:16:53.047328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:16:53.047632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:16:53.047901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:16:53.050139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:16:53.158163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:16:53.160217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622058d-dfbe-4160-8ffc-4867742e294f service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:16:53.178735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 742/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:16:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:02.829602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=22 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:03.176499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=66 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:12.927267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ee5a-6368-4830-ae3e-697aa310a4b9 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:12.927814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ee5a-6368-4830-ae3e-697aa310a4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:12.927814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211ee5a-6368-4830-ae3e-697aa310a4b9 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:12.929096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 729/1471] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:12.934239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f2dc-d602-4621-9df9-de7129158b20 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:12.934634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f2dc-d602-4621-9df9-de7129158b20 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:12.934790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f2dc-d602-4621-9df9-de7129158b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:13.001209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5024f2dc-d602-4621-9df9-de7129158b20 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:13.002860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5024f2dc-d602-4621-9df9-de7129158b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:13.099382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 743/1472] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:17:13.182454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122bca2-8bfe-460b-b305-caf6cb07920d None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:13.182678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122bca2-8bfe-460b-b305-caf6cb07920d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:13.182949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122bca2-8bfe-460b-b305-caf6cb07920d None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:13.184332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 730/1473] 10.222.0.13 () {54 vars in 967 bytes} [Wed Aug 10 11:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:13.189108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86ced-c567-4feb-a8fa-a5c2521d857c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:13.189942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86ced-c567-4feb-a8fa-a5c2521d857c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:13.190217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b86ced-c567-4feb-a8fa-a5c2521d857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:13.263954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-91b86ced-c567-4feb-a8fa-a5c2521d857c None None] MFA Rules not processed for user `3a95ab030c9940c9946c64c82567773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:13.265172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-91b86ced-c567-4feb-a8fa-a5c2521d857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:13.285692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:13.341253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 744/1474] 10.222.0.13 () {58 vars in 1064 bytes} [Wed Aug 10 11:17:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:13.348978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122bca2-8bfe-460b-b305-caf6cb07920d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:13.350536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c122bca2-8bfe-460b-b305-caf6cb07920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:13.454609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122bca2-8bfe-460b-b305-caf6cb07920d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:13.526925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:13.527525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:13.527802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:13.528103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:13.529700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:13.583916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:13.638378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:13.639801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a88556-5a82-446a-9907-43d314e68b4b service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:13.658461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 731/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:15.760264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f00f59-0620-4c37-86a6-3c33155511b6 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:15.760844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f00f59-0620-4c37-86a6-3c33155511b6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:15.760844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f00f59-0620-4c37-86a6-3c33155511b6 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:15.762351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 745/1476] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:15.768309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e918f-f5dc-40ec-aca6-2dbcf76ec3eb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:15.771635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e918f-f5dc-40ec-aca6-2dbcf76ec3eb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:15.771635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e918f-f5dc-40ec-aca6-2dbcf76ec3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:15.836866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cf1e918f-f5dc-40ec-aca6-2dbcf76ec3eb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:15.838071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cf1e918f-f5dc-40ec-aca6-2dbcf76ec3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:15.932701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 732/1477] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:15.956548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f00f59-0620-4c37-86a6-3c33155511b6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:15.958119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-57f00f59-0620-4c37-86a6-3c33155511b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:16.083228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f00f59-0620-4c37-86a6-3c33155511b6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:16.131216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:16.176910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:16.177527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:16.177827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:16.178090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:16.178712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:16.180437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:17:16.180437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:17:16.180437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:17:16.181552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f07af-b3ae-40b8-af3a-ce45d7d688e7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:16.200778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 746/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Wed Aug 10 11:17:15 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:17:18.267247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60e8e-2891-4e2f-8c6a-7532b346833a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:18.268553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60e8e-2891-4e2f-8c6a-7532b346833a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:18.268553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b60e8e-2891-4e2f-8c6a-7532b346833a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:18.269702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 733/1479] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:18.274992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8bc83-ca2c-4513-a278-ef976a18722d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:18.275398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8bc83-ca2c-4513-a278-ef976a18722d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:18.275566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8bc83-ca2c-4513-a278-ef976a18722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:18.341885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-98e8bc83-ca2c-4513-a278-ef976a18722d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:18.343129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-98e8bc83-ca2c-4513-a278-ef976a18722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:18.438791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 747/1480] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:18.482337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b60e8e-2891-4e2f-8c6a-7532b346833a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:18.483908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-71b60e8e-2891-4e2f-8c6a-7532b346833a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:18.493669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:18.628648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b60e8e-2891-4e2f-8c6a-7532b346833a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:18.706865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:18.739357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:18.740154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:18.740434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:18.740708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:18.742300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:18.833582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:18.835161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af2798-9f0a-4759-8789-bd9df03c1453 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:18.853835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 734/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:21.054346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfbfb3-810e-4cb5-8490-6aac2fc3520b None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:21.054874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfbfb3-810e-4cb5-8490-6aac2fc3520b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:21.054874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfbfb3-810e-4cb5-8490-6aac2fc3520b None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:21.056175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 748/1482] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:21.061519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e468-0b29-4303-8cf6-64651d6b1f61 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:21.061933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e468-0b29-4303-8cf6-64651d6b1f61 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:21.062059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e468-0b29-4303-8cf6-64651d6b1f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:21.128206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0643e468-0b29-4303-8cf6-64651d6b1f61 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:21.130511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0643e468-0b29-4303-8cf6-64651d6b1f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:21.175081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:21.226191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 735/1483] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:21.263539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfbfb3-810e-4cb5-8490-6aac2fc3520b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:21.265112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d1cfbfb3-810e-4cb5-8490-6aac2fc3520b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:21.288359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:21.368896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfbfb3-810e-4cb5-8490-6aac2fc3520b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:21.443687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:21.444328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:21.444609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:21.444882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:21.446494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:21.521466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:21.557920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:21.559363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28f08d3-caf4-489c-98ac-412f656a017e service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:21.577057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 749/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:23.646493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36efc6-e476-42f1-8744-a149c7ce03cb None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:23.647091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36efc6-e476-42f1-8744-a149c7ce03cb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:23.647091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36efc6-e476-42f1-8744-a149c7ce03cb None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:23.648538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 736/1485] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:23.653642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66db1-84d6-4a1b-bcaf-d7128cb8c997 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:23.654096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66db1-84d6-4a1b-bcaf-d7128cb8c997 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:23.654205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad66db1-84d6-4a1b-bcaf-d7128cb8c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:23.718362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-cad66db1-84d6-4a1b-bcaf-d7128cb8c997 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:23.719539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-cad66db1-84d6-4a1b-bcaf-d7128cb8c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:23.813286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 750/1486] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:23.852398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36efc6-e476-42f1-8744-a149c7ce03cb None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:23.854010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-df36efc6-e476-42f1-8744-a149c7ce03cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:23.956920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36efc6-e476-42f1-8744-a149c7ce03cb None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:24.028814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:24.030612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:24.031214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:24.031494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:24.031769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:24.033474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:24.141800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:24.143219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2f5f0-2c46-4533-9405-19c4df162e79 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:24.160810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 737/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:26.337681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371f61-5d40-4331-8aba-03622650915b None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:26.338284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371f61-5d40-4331-8aba-03622650915b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:26.338284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371f61-5d40-4331-8aba-03622650915b None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:26.339610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 751/1488] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:26.344794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc0949-28e4-40d2-9066-84621507dd40 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:26.345266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc0949-28e4-40d2-9066-84621507dd40 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:26.345373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc0949-28e4-40d2-9066-84621507dd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:26.414343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4cfc0949-28e4-40d2-9066-84621507dd40 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:26.415561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4cfc0949-28e4-40d2-9066-84621507dd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:26.469358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:26.514401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 738/1489] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:26.556259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5371f61-5d40-4331-8aba-03622650915b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:26.557795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a5371f61-5d40-4331-8aba-03622650915b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:26.594355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:26.663883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5371f61-5d40-4331-8aba-03622650915b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:26.736546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:26.737139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:26.737409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:26.737674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:26.739721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:26.818988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:26.852268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:26.853701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b946f07-7ad6-4420-b0fc-144df00e4e36 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:26.875069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 752/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:29.361071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26473362-ae26-4f49-a68e-b394aa0fd6d0 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:29.361664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26473362-ae26-4f49-a68e-b394aa0fd6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:29.361774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26473362-ae26-4f49-a68e-b394aa0fd6d0 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:29.363021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 739/1491] 10.222.0.13 () {54 vars in 927 bytes} [Wed Aug 10 11:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:17:29.368622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d842cd-0467-453c-8efe-70c08f4f16a3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:29.370358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d842cd-0467-453c-8efe-70c08f4f16a3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:29.370358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d842cd-0467-453c-8efe-70c08f4f16a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:29.435038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-35d842cd-0467-453c-8efe-70c08f4f16a3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:17:29.436283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-35d842cd-0467-453c-8efe-70c08f4f16a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:29.532056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 753/1492] 10.222.0.13 () {58 vars in 1024 bytes} [Wed Aug 10 11:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:17:29.571743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26473362-ae26-4f49-a68e-b394aa0fd6d0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:17:29.573339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-26473362-ae26-4f49-a68e-b394aa0fd6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:29.677227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26473362-ae26-4f49-a68e-b394aa0fd6d0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:17:29.748607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:17:29.749212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:17:29.749492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:17:29.749756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:17:29.751330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:17:29.760972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:29.859324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:17:29.860777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500c921-465d-4483-8218-760fd46b76e1 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:17:29.878786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 740/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:17:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:17:39.530233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=82 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:17:39.876555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=50 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:56.422679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:56.423204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:56.423204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:56.499037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:56.500261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:56.596331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 754/1494] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:56.604591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:56.605080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:56.605182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:56.680649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:56.681849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:56.775806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 741/1495] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:56.780536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:56.781725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:56.793487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8eb2ce-334c-40da-829d-1e7a5f7379b8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:56.853864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:56.888903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:56.889499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:56.889770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:56.890034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:56.890660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:18:56.890957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:56.891950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:56.891950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:18:56.891950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:18:56.892861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:56.904435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 755/1496] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:18:56.912870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:56.921960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af468f6-5a09-4d70-8a52-f88818994ef2 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:56.982376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:57.017097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:57.017698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:57.017976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:57.018243 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:57.018872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:57.020688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:57.020688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:18:57.020688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:18:57.029702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:57.140633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 742/1497] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:18:56 2022] POST /identity/v3/projects => generated 377 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:18:57.144783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:57.146394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:57.179357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf15600-7bdb-4a3a-ae96-a56b5aa6ea1a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:57.270708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:57.271250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:57.271522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:57.271784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:57.272429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:57.273422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:57.273422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:18:57.273422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:18:57.274601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:57.388085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 756/1498] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:18:57 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:18:57.389378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:57.390948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:57.452498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f69a05-595f-4adf-bcd5-aafa086a1bac admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:57.534383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:57.545485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:57.546077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:57.546349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:57.546611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:57.547239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:57.548264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:57.548264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:18:57.548264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:18:57.549142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:57.562512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 743/1499] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:18:57.566770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:57.568754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:57.580268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a0a33-1ae7-4957-8e44-3488ce9a81aa admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:57.605625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:57.673701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:57.674316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:57.674775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:57.675653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:57.676490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:57.678144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:57.678144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:18:57.678144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:18:57.679016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44724ea8-6fc8-44bf-b5a3-9054b519a711 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:57.691546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 757/1500] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:18:57.696207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:57.697822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:57.731203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5599fb7-b13d-4997-bdcd-5ed5f2a248b0 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:57.822995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:57.823646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:57.823923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:57.824228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] PATH_INFO: `/v3/projects/f338b37fbb7f46aea12735194d9b36fe/users/e0e72381540946ceac24e33ada518047/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:57.846112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f338b37fbb7f46aea12735194d9b36fe, user_id=e0e72381540946ceac24e33ada518047, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:57.847133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:57.847133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:18:57.847133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:18:57.849706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbdaa6-f385-416b-af59-c2cce52f3567 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:57.955449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 744/1501] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:18:57 2022] PUT /identity/v3/projects/f338b37fbb7f46aea12735194d9b36fe/users/e0e72381540946ceac24e33ada518047/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:18:57.968523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c50fe-6274-4d56-99d0-230a8a3cdab7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:57.968806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c50fe-6274-4d56-99d0-230a8a3cdab7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:57.969076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c50fe-6274-4d56-99d0-230a8a3cdab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:58.025547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c79c50fe-6274-4d56-99d0-230a8a3cdab7 None None] MFA Rules not processed for user `e0e72381540946ceac24e33ada518047`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:58.026758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c79c50fe-6274-4d56-99d0-230a8a3cdab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:58.104555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 758/1502] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:18:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:18:58.118029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33949bf4-3371-4850-87be-492affc94b7a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:58.118309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33949bf4-3371-4850-87be-492affc94b7a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:58.118577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33949bf4-3371-4850-87be-492affc94b7a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:58.119910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 745/1503] 10.222.0.13 () {54 vars in 964 bytes} [Wed Aug 10 11:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:18:58.124593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300728-9fa6-4033-8bb2-85cf25351991 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:58.124877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300728-9fa6-4033-8bb2-85cf25351991 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:58.125261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70300728-9fa6-4033-8bb2-85cf25351991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:58.156885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:58.201660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-70300728-9fa6-4033-8bb2-85cf25351991 None None] MFA Rules not processed for user `3cada6c06ef8427395eda888397efac4`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:58.202863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-70300728-9fa6-4033-8bb2-85cf25351991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:58.277666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 759/1504] 10.222.0.13 () {58 vars in 1061 bytes} [Wed Aug 10 11:18:58 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:58.285516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33949bf4-3371-4850-87be-492affc94b7a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:58.287068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-33949bf4-3371-4850-87be-492affc94b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:58.336950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:58.390926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33949bf4-3371-4850-87be-492affc94b7a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:58.464533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:58.465131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:58.465409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:58.465678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:58.467257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:58.563640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:58.565117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:58.569215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:58.584625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 746/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:18:58.629345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec47576-a84c-44db-9bbd-b4cdc2bddedd None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:58.629772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec47576-a84c-44db-9bbd-b4cdc2bddedd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:58.629887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec47576-a84c-44db-9bbd-b4cdc2bddedd None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:58.631809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 760/1506] 10.222.0.13 () {52 vars in 830 bytes} [Wed Aug 10 11:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:18:58.635339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:58.636578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:58.738407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcdf1bb-0ca2-4f42-89c8-2a44b3ce8c5d service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:58.831584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:58.832329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:58.832608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:58.832879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] PATH_INFO: `/v3/users/e0e72381540946ceac24e33ada518047` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:58.845162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e72381540946ceac24e33ada518047)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:58.846182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:18:58.846182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:18:58.846182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:18:58.847387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:58.857457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:59.015472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e72381540946ceac24e33ada518047'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:18:59.015832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e72381540946ceac24e33ada518047'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:18:59.016389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e72381540946ceac24e33ada518047'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:18:59.030414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e72381540946ceac24e33ada518047'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:18:59.097304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e72381540946ceac24e33ada518047'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:18:59.126761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 747/1507] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:18:58 2022] DELETE /identity/v3/users/e0e72381540946ceac24e33ada518047 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:18:59.131279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:59.131562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:59.131834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:59.187459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:18:59.188679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:59.263549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:59.276570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 761/1508] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:18:59.286161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:18:59.295981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:59.386758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991100d-dc3e-4615-9f59-0e91c4e68e48 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:18:59.441054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:18:59.460151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:18:59.460763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:18:59.461043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:18:59.461317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:18:59.462908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:18:59.572321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:18:59.573754 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05085b91-a132-4636-9d32-f77dfbaf18b0 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:18:59.595260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 748/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:19:00.089452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:19:00.090714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:19:00.102144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01197f52-524f-4b86-990b-885ec6b9a2ff None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:19:00.203081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:19:00.203718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:19:00.204014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:19:00.204301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] PATH_INFO: `/v3/projects/f338b37fbb7f46aea12735194d9b36fe` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:19:00.214194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f338b37fbb7f46aea12735194d9b36fe)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:19:00.215219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:19:00.215219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:19:00.215219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:19:00.216443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:19:00.319336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f338b37fbb7f46aea12735194d9b36fe', 'request_id': 'req-519e9d3e-d7bb-426e-95ca-2b82d178cf73'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:19:00.326284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f338b37fbb7f46aea12735194d9b36fe', 'request_id': 'req-519e9d3e-d7bb-426e-95ca-2b82d178cf73'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:19:00.338986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f338b37fbb7f46aea12735194d9b36fe', 'request_id': 'req-519e9d3e-d7bb-426e-95ca-2b82d178cf73'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:19:00.430333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] The token cache is being invalidate because project f338b37fbb7f46aea12735194d9b36fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:19:00.430675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:19:00.431107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-519e9d3e-d7bb-426e-95ca-2b82d178cf73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:19:00.432549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 762/1510] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:19:00 2022] DELETE /identity/v3/projects/f338b37fbb7f46aea12735194d9b36fe => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:19:09.590001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=62 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:19:10.428268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:29.424605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64927-bca4-48f5-994f-be168bbd94c1 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.425160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64927-bca4-48f5-994f-be168bbd94c1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.425160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d64927-bca4-48f5-994f-be168bbd94c1 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.426479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:29.431319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9299c-4ff3-43c3-a44f-afa82c177571 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.431754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9299c-4ff3-43c3-a44f-afa82c177571 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.431910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9299c-4ff3-43c3-a44f-afa82c177571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.512868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-13d9299c-4ff3-43c3-a44f-afa82c177571 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:29.513979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-13d9299c-4ff3-43c3-a44f-afa82c177571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:29.618246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:29.774741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64927-bca4-48f5-994f-be168bbd94c1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:29.780634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d9d64927-bca4-48f5-994f-be168bbd94c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:29.892393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d64927-bca4-48f5-994f-be168bbd94c1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:29.897904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a611d22-331d-457b-b8cb-601bc5239f2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:29.899131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a611d22-331d-457b-b8cb-601bc5239f2d service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:29.899415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a611d22-331d-457b-b8cb-601bc5239f2d service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:29.899691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a611d22-331d-457b-b8cb-601bc5239f2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:29.907636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a611d22-331d-457b-b8cb-601bc5239f2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:29.909237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a611d22-331d-457b-b8cb-601bc5239f2d service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:29.918989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 750/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:30.762685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2162eec-57ec-4fbd-a1b4-6dd7c1b65562 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.763136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2162eec-57ec-4fbd-a1b4-6dd7c1b65562 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.763241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2162eec-57ec-4fbd-a1b4-6dd7c1b65562 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.764646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:30.769348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07b4b-fbf6-4c56-98ab-efaafecd7bfa None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.769770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07b4b-fbf6-4c56-98ab-efaafecd7bfa None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.769876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda07b4b-fbf6-4c56-98ab-efaafecd7bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.789650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-fda07b4b-fbf6-4c56-98ab-efaafecd7bfa None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:30.790880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fda07b4b-fbf6-4c56-98ab-efaafecd7bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:30.798627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:30.950930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2162eec-57ec-4fbd-a1b4-6dd7c1b65562 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:30.965016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2162eec-57ec-4fbd-a1b4-6dd7c1b65562 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:30.970371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16ffdf8-b579-43be-acef-16ca3ac06876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:30.970958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ffdf8-b579-43be-acef-16ca3ac06876 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:30.971231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ffdf8-b579-43be-acef-16ca3ac06876 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:30.971497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16ffdf8-b579-43be-acef-16ca3ac06876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:30.980061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ffdf8-b579-43be-acef-16ca3ac06876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:30.981726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16ffdf8-b579-43be-acef-16ca3ac06876 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:30.990791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 765/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:31.541178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ab97-ee6a-4412-9111-0fa652d799e3 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:31.541465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ab97-ee6a-4412-9111-0fa652d799e3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:31.541743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ab97-ee6a-4412-9111-0fa652d799e3 None None] PATH_INFO: `/v3` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:31.543017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 752/1517] 10.222.0.45 () {54 vars in 925 bytes} [Wed Aug 10 11:20:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:20:31.549053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7fc3e-2f37-401c-9251-9563185eadf3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:31.549335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7fc3e-2f37-401c-9251-9563185eadf3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:31.549614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7fc3e-2f37-401c-9251-9563185eadf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:31.611689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6dd7fc3e-2f37-401c-9251-9563185eadf3 None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:31.612911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6dd7fc3e-2f37-401c-9251-9563185eadf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:31.629561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:31.684512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 766/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 10 11:20:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:40.784848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=56 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:40.805990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb18515-5365-4605-bdb1-941bc2b68d7d None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:40.806367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb18515-5365-4605-bdb1-941bc2b68d7d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:40.806704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb18515-5365-4605-bdb1-941bc2b68d7d None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:40.808086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 753/1519] 10.222.0.13 () {54 vars in 961 bytes} [Wed Aug 10 11:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:40.813220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089fc57-c4f7-44ab-86c1-623d77fe37ec None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:40.813663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089fc57-c4f7-44ab-86c1-623d77fe37ec None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:40.813765 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089fc57-c4f7-44ab-86c1-623d77fe37ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:40.832965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d089fc57-c4f7-44ab-86c1-623d77fe37ec None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:40.834178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d089fc57-c4f7-44ab-86c1-623d77fe37ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:40.843335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 767/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Wed Aug 10 11:20:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:40.850757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb18515-5365-4605-bdb1-941bc2b68d7d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:40.864663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb18515-5365-4605-bdb1-941bc2b68d7d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:40.869814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413b369-e0c5-4a6f-8998-d3cc81395872 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:40.870394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413b369-e0c5-4a6f-8998-d3cc81395872 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:40.870675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413b369-e0c5-4a6f-8998-d3cc81395872 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:40.870950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413b369-e0c5-4a6f-8998-d3cc81395872 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:40.878980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413b369-e0c5-4a6f-8998-d3cc81395872 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:40.880582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8413b369-e0c5-4a6f-8998-d3cc81395872 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:40.890625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 754/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:20:42.235080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236dcfa-7a29-4a43-a90b-2190636f03f8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:42.235647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236dcfa-7a29-4a43-a90b-2190636f03f8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:42.235647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236dcfa-7a29-4a43-a90b-2190636f03f8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:42.236973 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:42.241622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efa0e1-bec6-44ea-bc6c-0a0ec3df7ea8 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:42.242018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efa0e1-bec6-44ea-bc6c-0a0ec3df7ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:42.242268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efa0e1-bec6-44ea-bc6c-0a0ec3df7ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:42.262378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a1efa0e1-bec6-44ea-bc6c-0a0ec3df7ea8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:42.263580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a1efa0e1-bec6-44ea-bc6c-0a0ec3df7ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:42.271329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:42.424987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236dcfa-7a29-4a43-a90b-2190636f03f8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:42.439857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236dcfa-7a29-4a43-a90b-2190636f03f8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:42.445233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db33e7-6a71-44e9-aea4-e1553c6e3dc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:42.445826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db33e7-6a71-44e9-aea4-e1553c6e3dc4 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:42.446103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db33e7-6a71-44e9-aea4-e1553c6e3dc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:42.446669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db33e7-6a71-44e9-aea4-e1553c6e3dc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:42.454756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db33e7-6a71-44e9-aea4-e1553c6e3dc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:42.456364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db33e7-6a71-44e9-aea4-e1553c6e3dc4 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:42.465709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 769/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:43.292117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6406-2cb2-43a5-a56a-0efd7b6de127 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:43.292691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6406-2cb2-43a5-a56a-0efd7b6de127 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:43.292691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f6406-2cb2-43a5-a56a-0efd7b6de127 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:43.294005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 10 11:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:43.298968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b94a2-3cbe-46ec-957d-e4bb366dc9d2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:43.299405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b94a2-3cbe-46ec-957d-e4bb366dc9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:43.299503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b94a2-3cbe-46ec-957d-e4bb366dc9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:43.318477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ee9b94a2-3cbe-46ec-957d-e4bb366dc9d2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:43.319663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ee9b94a2-3cbe-46ec-957d-e4bb366dc9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:43.328272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 10 11:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:43.480555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f6406-2cb2-43a5-a56a-0efd7b6de127 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:43.495139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f6406-2cb2-43a5-a56a-0efd7b6de127 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:43.500491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e6758-cb8b-45ef-a662-ff6c12be8b18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:43.501068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e6758-cb8b-45ef-a662-ff6c12be8b18 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:43.501341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e6758-cb8b-45ef-a662-ff6c12be8b18 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:43.501605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e6758-cb8b-45ef-a662-ff6c12be8b18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:43.510455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e6758-cb8b-45ef-a662-ff6c12be8b18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:43.512020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191e6758-cb8b-45ef-a662-ff6c12be8b18 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:43.521591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 757/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:47.615003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45037-3778-4dde-a2c7-b5951d68f5e9 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.615552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45037-3778-4dde-a2c7-b5951d68f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.615552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d45037-3778-4dde-a2c7-b5951d68f5e9 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.616963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:47.622601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fe347-9a15-4ecb-951c-6ec895d55fc7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.623018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fe347-9a15-4ecb-951c-6ec895d55fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.623130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14fe347-9a15-4ecb-951c-6ec895d55fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.642685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f14fe347-9a15-4ecb-951c-6ec895d55fc7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:47.643866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f14fe347-9a15-4ecb-951c-6ec895d55fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.652528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:47.726786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45037-3778-4dde-a2c7-b5951d68f5e9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:47.728554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-13d45037-3778-4dde-a2c7-b5951d68f5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:47.806788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d45037-3778-4dde-a2c7-b5951d68f5e9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:47.810388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bc8ae-f838-4678-8bda-dc3f307878a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:47.810991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bc8ae-f838-4678-8bda-dc3f307878a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:47.811268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bc8ae-f838-4678-8bda-dc3f307878a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:47.811561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bc8ae-f838-4678-8bda-dc3f307878a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:47.820346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711bc8ae-f838-4678-8bda-dc3f307878a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:47.822611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711bc8ae-f838-4678-8bda-dc3f307878a0 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:47.831785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 772/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:49.813822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df365e8-ebc5-438c-a1ea-8773d6fa73a6 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.814366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df365e8-ebc5-438c-a1ea-8773d6fa73a6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.814483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df365e8-ebc5-438c-a1ea-8773d6fa73a6 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.815790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:49.821025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341792-dd30-437d-bc28-c27080f07a49 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.821713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341792-dd30-437d-bc28-c27080f07a49 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.821858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5341792-dd30-437d-bc28-c27080f07a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.841550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c5341792-dd30-437d-bc28-c27080f07a49 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:49.842721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c5341792-dd30-437d-bc28-c27080f07a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:49.850314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:20:49.886177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df365e8-ebc5-438c-a1ea-8773d6fa73a6 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:49.896811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df365e8-ebc5-438c-a1ea-8773d6fa73a6 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:49.900909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7aa7c-6df4-4a80-aa3c-a8e75a2118e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:49.901710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7aa7c-6df4-4a80-aa3c-a8e75a2118e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:49.901983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7aa7c-6df4-4a80-aa3c-a8e75a2118e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:49.902246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7aa7c-6df4-4a80-aa3c-a8e75a2118e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:49.911181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7aa7c-6df4-4a80-aa3c-a8e75a2118e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:49.912622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7aa7c-6df4-4a80-aa3c-a8e75a2118e6 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:49.921783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 760/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:53.927102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a6e03-11eb-458b-875e-171692d6a67c None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:53.927673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a6e03-11eb-458b-875e-171692d6a67c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:53.927673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a6e03-11eb-458b-875e-171692d6a67c None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:53.929014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:53.934700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90329-426c-46f1-8c6d-44c618aca16c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:53.935138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90329-426c-46f1-8c6d-44c618aca16c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:53.935246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde90329-426c-46f1-8c6d-44c618aca16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:53.954502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cde90329-426c-46f1-8c6d-44c618aca16c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:53.955720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cde90329-426c-46f1-8c6d-44c618aca16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:53.963950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:54.272650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a6e03-11eb-458b-875e-171692d6a67c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:54.274311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-918a6e03-11eb-458b-875e-171692d6a67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:54.339496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:20:54.352358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a6e03-11eb-458b-875e-171692d6a67c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:54.356633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c45571-3954-44b7-b4ef-1a56f724d086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:54.357264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c45571-3954-44b7-b4ef-1a56f724d086 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:54.357538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c45571-3954-44b7-b4ef-1a56f724d086 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:54.357807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c45571-3954-44b7-b4ef-1a56f724d086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:54.367158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c45571-3954-44b7-b4ef-1a56f724d086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:54.368643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c45571-3954-44b7-b4ef-1a56f724d086 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:54.378531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 775/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:20:56.408239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0cb7-e494-4903-95cb-16e0b2dd9381 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:56.408803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0cb7-e494-4903-95cb-16e0b2dd9381 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:56.408803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0cb7-e494-4903-95cb-16e0b2dd9381 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:56.410119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 10 11:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:20:56.415122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed92ac3-3825-48b9-8488-ede47eb276e4 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:56.415599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed92ac3-3825-48b9-8488-ede47eb276e4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:56.416288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed92ac3-3825-48b9-8488-ede47eb276e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:56.435104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-aed92ac3-3825-48b9-8488-ede47eb276e4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:20:56.436329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-aed92ac3-3825-48b9-8488-ede47eb276e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:56.444015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 10 11:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:20:56.751013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0cb7-e494-4903-95cb-16e0b2dd9381 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:20:56.752725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ed4b0cb7-e494-4903-95cb-16e0b2dd9381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:20:56.766330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0cb7-e494-4903-95cb-16e0b2dd9381 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:20:56.769826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b507f3f-adee-4c31-a880-53dee4b755b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:20:56.770412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507f3f-adee-4c31-a880-53dee4b755b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:20:56.770690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507f3f-adee-4c31-a880-53dee4b755b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:20:56.770962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b507f3f-adee-4c31-a880-53dee4b755b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:20:56.780402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507f3f-adee-4c31-a880-53dee4b755b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:20:56.781879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b507f3f-adee-4c31-a880-53dee4b755b0 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:20:56.791290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 763/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:20:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:21:06.430461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=11 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:21:06.786969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=33 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:22.462580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df369d-4dd5-4af8-8ff1-49847c77db4b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.463198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df369d-4dd5-4af8-8ff1-49847c77db4b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.463198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df369d-4dd5-4af8-8ff1-49847c77db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.486977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-17df369d-4dd5-4af8-8ff1-49847c77db4b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.488340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-17df369d-4dd5-4af8-8ff1-49847c77db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.503643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.504334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 777/1540] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.504433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.504433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.515474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aeee03-8916-4d44-bd71-e72e058f1e76 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.515844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aeee03-8916-4d44-bd71-e72e058f1e76 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.516033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aeee03-8916-4d44-bd71-e72e058f1e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.537237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.538482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.544953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-14aeee03-8916-4d44-bd71-e72e058f1e76 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.546915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-14aeee03-8916-4d44-bd71-e72e058f1e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.559173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 778/1541] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:22.560929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46523582-840f-4851-8bf6-0648fe3ebfc3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.561894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46523582-840f-4851-8bf6-0648fe3ebfc3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.562271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46523582-840f-4851-8bf6-0648fe3ebfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.565872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 764/1542] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.566724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:22.581401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ccfc98-e9fd-4096-9af0-03e3e0ed7e5f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:22.593227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-46523582-840f-4851-8bf6-0648fe3ebfc3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.594446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-46523582-840f-4851-8bf6-0648fe3ebfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.596695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:22.597319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.597599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.597871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.598528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:22.598829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:22.599842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:22:22.599842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:22:22.599842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:22:22.600886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db5e2-4b78-44b6-8996-68e060ab36a2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:22.608736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 779/1543] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.610165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5117ea0-1cf5-4073-8ace-beb5ac399a32 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.610451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5117ea0-1cf5-4073-8ace-beb5ac399a32 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.610725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5117ea0-1cf5-4073-8ace-beb5ac399a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.622164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 765/1544] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:22.623440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.623891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.629070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.638997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b5117ea0-1cf5-4073-8ace-beb5ac399a32 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.640303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b5117ea0-1cf5-4073-8ace-beb5ac399a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.648981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 780/1545] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.650287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705064d-98d4-47bb-85dd-103f3f296274 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.650574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705064d-98d4-47bb-85dd-103f3f296274 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.650848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705064d-98d4-47bb-85dd-103f3f296274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.654511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.656608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.671287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 766/1546] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.672175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:22.673553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5705064d-98d4-47bb-85dd-103f3f296274 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.674815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5705064d-98d4-47bb-85dd-103f3f296274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.682015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008ce7e-4595-4112-b741-5ea9133dfc9c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:22.692489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 781/1547] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.692489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd8ee6-7cd1-4147-8ec6-00c64c43d6b5 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.692489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd8ee6-7cd1-4147-8ec6-00c64c43d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.692489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd8ee6-7cd1-4147-8ec6-00c64c43d6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.692837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:22.692837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.692837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.692837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.693160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:22.693160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:22.708339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f5dd8ee6-7cd1-4147-8ec6-00c64c43d6b5 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.709742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f5dd8ee6-7cd1-4147-8ec6-00c64c43d6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.732311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 782/1548] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.732311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca006-19d9-490a-9721-7b334e190180 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.732311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca006-19d9-490a-9721-7b334e190180 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.732311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feca006-19d9-490a-9721-7b334e190180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.739534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1feca006-19d9-490a-9721-7b334e190180 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.740814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1feca006-19d9-490a-9721-7b334e190180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.748901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 783/1549] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.749783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca006-19d9-490a-9721-7b334e190180 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:22.761271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feca006-19d9-490a-9721-7b334e190180 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:22.766705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:22.767305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.767572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.767847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.768515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:22.768819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:22.770436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e668589-a93c-44e4-a872-c53a83865d1a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:22.782986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 784/1550] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:22.784518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b1d7-0288-49c4-ab95-a1d6989c13af None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.784518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b1d7-0288-49c4-ab95-a1d6989c13af None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.784773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793b1d7-0288-49c4-ab95-a1d6989c13af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.804854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c793b1d7-0288-49c4-ab95-a1d6989c13af None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.806047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c793b1d7-0288-49c4-ab95-a1d6989c13af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.814080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 785/1551] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.815223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7c5e0-fc82-46cb-977d-85cf4d4a284b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.815502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7c5e0-fc82-46cb-977d-85cf4d4a284b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.815774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7c5e0-fc82-46cb-977d-85cf4d4a284b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.833693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 767/1552] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:22.833693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:22.835705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b0f7c5e0-fc82-46cb-977d-85cf4d4a284b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.836927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0f7c5e0-fc82-46cb-977d-85cf4d4a284b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.894273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98884085-12cc-4e6a-b352-f3647a932984 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:22.895173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 786/1553] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.896555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce942b-a41c-4f23-8bf4-23b1714433ea None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.896835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce942b-a41c-4f23-8bf4-23b1714433ea None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.897105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ce942b-a41c-4f23-8bf4-23b1714433ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.899456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:22.900167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.900510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.900842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.901559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:22.901927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:22.903576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6c610-eca4-473a-bd72-c7f0ee9a1116 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:22.923501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 768/1554] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:22.923743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-68ce942b-a41c-4f23-8bf4-23b1714433ea None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.923861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.923961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-68ce942b-a41c-4f23-8bf4-23b1714433ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.924097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.924097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.926439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 787/1555] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.928399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cd1fd-690f-4db8-96fc-35149805d429 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.928399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cd1fd-690f-4db8-96fc-35149805d429 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.928399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98cd1fd-690f-4db8-96fc-35149805d429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.939554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.940787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.946995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f98cd1fd-690f-4db8-96fc-35149805d429 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.948412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f98cd1fd-690f-4db8-96fc-35149805d429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.949189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 769/1556] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.949766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:22.959955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e089a7-6136-400f-bccc-4def65702c67 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:22.960655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 788/1557] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.961873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.962151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.962487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.965553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:22.966343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:22.966778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:22.967432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:22.968686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:22.970519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:22.982189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:22.983794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:22.991514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 789/1558] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:22.992286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.001489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b012ed-bebc-454b-9713-754b882f4e74 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.005950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.006521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.006795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.007062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.007664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.009614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.098735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 770/1559] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.102407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.129457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 790/1560] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:22 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.133041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.140612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d933c52-6d06-43ad-95f5-6d141fab7e0b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.167018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e838d8c5-5ca6-4010-b523-97c41fb911c5 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.172158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.172746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.173032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.173307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.173957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:23.174259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.176030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.187951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 771/1561] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:23.188795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.193802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:23.194666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:23.199555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e950d-fa3d-42b1-aa83-ed7173ae5065 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.205194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.205868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.206083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.206353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.206684 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.206800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.207005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.207273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.207434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:23.207740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.207892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.208929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:22:23.208929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:22:23.208929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:22:23.210161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.210255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.228518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 772/1562] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:23.228518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.231788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a37920-9710-4c73-a6d1-78e17f6c25fe admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.237034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.237558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.237833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.238104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.238736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:23.239035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.240643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9541343-e71c-415a-80dc-4ffbc263423e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.252527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 773/1563] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:23.253650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.253934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.254208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.273222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:23.274412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:23.282881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 774/1564] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:23.283675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.292624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c9fde-067a-4fbc-a20d-1765df507a04 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.297351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.297924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.298179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.298444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.299059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:23.299351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.300935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.310011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 791/1565] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.310825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.320523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 775/1566] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:23.320523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.324384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162b0090-727b-4dcf-ba65-2de762c91f27 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.374574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76828e0-690c-4d5a-be53-eab412442002 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.379534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.380175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.383963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.383963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.383963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.383963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.384645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.385246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.385520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.385784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.386419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.388288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.402315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 776/1567] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:23.402315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.409647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def741-c98e-4cf4-983c-6ddd49ad86c3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.414134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.414694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.414967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.415361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.415954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.417900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.500234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 792/1568] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.500974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.510244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5e6b0-97c5-44c4-a301-f2dd67b206dd admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.514761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.515331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.515603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.515869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.516506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.518503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.527020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 777/1569] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.528410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.590767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805a6b8-70e9-4434-92e3-a1d3b98e6cc5 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.595875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.596499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.596775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.597041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.597650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.599565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.610389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 793/1570] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.614376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.675366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aa8e2-bfa3-4fbe-9ced-e5bdfd15cb4d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.680058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.680650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.680922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.681185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.681857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:23.682178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.683752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.695821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 794/1571] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:23.696629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.706361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7a0a9-9c35-40c9-8528-88c60537eb45 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.710042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 778/1572] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.710839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.714074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.714676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.714982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.715255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.715897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.717828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.730080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:23.772974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743e41e-b52a-447a-b270-d1e896bae194 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.777594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.778184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.778456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.778723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.779335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.781306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.814758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 795/1573] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.815613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.825395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd20f-cd52-40c1-8f44-eb20bce7b836 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.829971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.830552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.830832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.831088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.831713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.833344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.845307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 796/1574] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:23.846088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.856462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775fbba-f6c8-4fbf-9b86-fa96cbd23c8b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.861688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.862254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.862530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.862799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.863448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.865028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.874495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 797/1575] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:23.875523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.881379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:23.885131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e3222f-b1f9-4a28-a244-7ab6ee682ea2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.889842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.890413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.890691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.890960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.891570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.893522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:23.895373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 779/1576] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:23.896332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:23.958048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2b6ef-34f3-427e-99f7-256cd33779ea admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:23.962640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:23.963238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:23.963520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:23.963794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:23.964488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:23.965521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:22:23.965521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:22:23.965521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:22:23.966723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.011444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 798/1577] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:24.012377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.021700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109f163-2bca-45dd-81d3-c22a62c571f0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.027007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.027527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.027806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.028113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.028730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.031096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.074581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 780/1578] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:23 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:24.074581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.082922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade89020-d95b-4e4e-8593-f13ee75005ad admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.087432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.088010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.088296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.088562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.089159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.091068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.132846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 799/1579] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:24 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:24.133679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.193427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d7c32-1178-4273-8af7-9f0e2d575ec1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.197153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 781/1580] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:24 2022] POST /identity/v3/users => generated 289 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:24.198356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.201498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.201617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.201617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.201617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.201617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.202957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.209584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bccb1d-69fe-4c51-b2df-dc40cc138096 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.213411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 800/1581] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.214090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.214452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.215103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.215383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.215654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] PATH_INFO: `/v3/projects/ac78c80460af43e0aa28dc607af188d5/users/2386a816917a4c3d97f394b07d901386/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.224872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec0b47-62e7-4165-9058-9e72c7829329 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.229567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.230163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.230435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.230699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.231381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.232502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac78c80460af43e0aa28dc607af188d5, user_id=2386a816917a4c3d97f394b07d901386, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.233164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.235317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.242924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 801/1582] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.243844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.253414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3beded-ddf1-4281-a36b-13df1c56745b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.257801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.258407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.258675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.258960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.259528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.261401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.314178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 782/1583] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:24 2022] PUT /identity/v3/projects/ac78c80460af43e0aa28dc607af188d5/users/2386a816917a4c3d97f394b07d901386/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:24.315180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.347564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 802/1584] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:24 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:24.347564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.358472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc94e7-28c9-4f0d-8591-8830b8659536 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.363723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:24.381226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c0b4b-ef73-415d-8c4e-2cf30d7cdf00 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.385966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.386629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.386908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.387160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.387826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.390166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.400214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 783/1585] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.401056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.410995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed9777-7846-4c27-962d-0664b7640547 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.415493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b28a1-6525-4781-9eaf-c9bd57f1a1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.416100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b28a1-6525-4781-9eaf-c9bd57f1a1c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.416373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b28a1-6525-4781-9eaf-c9bd57f1a1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.416636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b28a1-6525-4781-9eaf-c9bd57f1a1c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.417238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b28a1-6525-4781-9eaf-c9bd57f1a1c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.419093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b28a1-6525-4781-9eaf-c9bd57f1a1c2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.420114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.421199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.421479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.421749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.422383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.424078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.434004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 803/1586] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.434802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.444737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed846fc-1127-43ca-80b8-2b0c85d32ca1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.449464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.450054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.450326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.450589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.451198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.453755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.463204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 804/1587] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.463991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.474034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637702c2-0135-4a6a-9ea7-9e1f4b6251b1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.478594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.479168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.479440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.479703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.480334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.481837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.491834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 805/1588] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.492592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.501843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e714b-cbf8-4fb0-9ea8-51ee87269f89 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.506496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.511296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.511296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.511296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] PATH_INFO: `/v3/projects/282c9e4b65d64085a0ba1a3d356f59fa/users/87adb637dd814cf8be9ff6c2a2bc26ea/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.511296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:24.522985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] RBAC: Authorizing `identity:create_grant(project_id=282c9e4b65d64085a0ba1a3d356f59fa, user_id=87adb637dd814cf8be9ff6c2a2bc26ea, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.523980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:22:24.523980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:22:24.523980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:22:24.525908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.534985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 784/1589] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:24 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:24.536415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.539837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.539837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.555808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:24.557051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:24.565291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 785/1590] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:24.566154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.577591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430792b-ef4b-465c-bce2-1b94910bf041 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.581244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.581831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.582132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.582401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.583036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.584682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.594576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 786/1591] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.595490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.604775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b126696-ceba-417e-b351-b53efd5657c1 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.609759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.611761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.611761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.611761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.611761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.612996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.622418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 787/1592] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.623234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.633487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de505e0-64c0-4bdb-9ff8-4f710ce45c66 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.640896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 806/1593] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:24 2022] PUT /identity/v3/projects/282c9e4b65d64085a0ba1a3d356f59fa/users/87adb637dd814cf8be9ff6c2a2bc26ea/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:24.641723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.693556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.698044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.698044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.698044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.698044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.698044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.708330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 788/1594] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.709119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.715385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c864548-1ad3-42c9-921f-bd409677a95f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.718851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441946c-b532-4525-86fc-5e16249daa59 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.720355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.720962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.721235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.721504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.722135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.723651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2cc6f-dbc2-44d6-8ac1-0bfa0354e906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.723794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.724341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2cc6f-dbc2-44d6-8ac1-0bfa0354e906 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.724619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2cc6f-dbc2-44d6-8ac1-0bfa0354e906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.724894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2cc6f-dbc2-44d6-8ac1-0bfa0354e906 admin admin] PATH_INFO: `/v3/projects/526e4ff2ae594d54a53e628df8802a92/users/9b5890ee9a114299b0a15a6608503632/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.733771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 807/1595] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.734591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.742094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2cc6f-dbc2-44d6-8ac1-0bfa0354e906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=526e4ff2ae594d54a53e628df8802a92, user_id=9b5890ee9a114299b0a15a6608503632, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.743881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca2e5f-1fbf-48ce-8b08-ec0e048d6fd8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.745122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2cc6f-dbc2-44d6-8ac1-0bfa0354e906 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.749118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.749728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.750033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.750305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.750948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.752554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.763587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 808/1596] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.763587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.765501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:24.822012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce83b-a5e7-490f-b3f5-8c2005cc0071 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.825827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.826427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.827296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.827797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.836509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.837922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.847456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 809/1597] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:24.848261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.856948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a582dd-bd8c-4e5c-b097-47fc207396f0 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.861566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.862143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.862417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.862681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.863295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.864950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.875487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 810/1598] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:24 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:24.876286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.885936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4163e4e-6071-4abc-adca-1acef0feef9a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.888715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 789/1599] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:24 2022] PUT /identity/v3/projects/526e4ff2ae594d54a53e628df8802a92/users/9b5890ee9a114299b0a15a6608503632/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:24.894535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.894818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.895299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.914332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:24.915564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:24.945731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.946396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.946686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.946944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] PATH_INFO: `/v3/projects/0b2a3c6102fb4fe5bd9c9ef1b8f8e78b/users/0782e27e2318438fbf155dfa9cb834b6/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:24.950282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:24.963969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2a3c6102fb4fe5bd9c9ef1b8f8e78b, user_id=0782e27e2318438fbf155dfa9cb834b6, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:24.966717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:24.981389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 790/1600] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:24.982240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:24.992960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781db0a4-5a84-4a58-a1b8-92f85c64b2eb None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:24.997622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:24.998278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:24.999452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:24.999452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] PATH_INFO: `/v3/projects/b50f3ae6a06943ee9cce4bb812cfa5d1/users/c735731e1e3c4fbca8fab3960dd4b4c8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.009023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:25.016378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50f3ae6a06943ee9cce4bb812cfa5d1, user_id=c735731e1e3c4fbca8fab3960dd4b4c8, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.018998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.066595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 811/1601] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:24 2022] PUT /identity/v3/projects/0b2a3c6102fb4fe5bd9c9ef1b8f8e78b/users/0782e27e2318438fbf155dfa9cb834b6/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:25.067563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.096894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 791/1602] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:24 2022] PUT /identity/v3/projects/b50f3ae6a06943ee9cce4bb812cfa5d1/users/c735731e1e3c4fbca8fab3960dd4b4c8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:25.097920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.131961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc071a0-8b51-4a05-9e2d-51d6cc9f7558 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.171375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938e631-4a41-4af1-94ff-2d2a21513562 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.176285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.176895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.177169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.177504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.178109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.179777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.191814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 792/1603] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:25.192863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.203335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3370de9-f07b-4fdd-98cb-52acb9725356 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.205439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac37062-b0fe-473e-8ec6-aa6dd993b47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.206154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac37062-b0fe-473e-8ec6-aa6dd993b47b admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.206395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac37062-b0fe-473e-8ec6-aa6dd993b47b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.206670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac37062-b0fe-473e-8ec6-aa6dd993b47b admin admin] PATH_INFO: `/v3/projects/362ec4487088496a8198a3cf71815256/users/11be37af210c45fc9ddebf7f28b265ef/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.209084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.209844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.210106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.210381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] PATH_INFO: `/v3/projects/f19ad3068694411f9808277a34994ea2/users/6fbb043908dd4807b55c4c063105c26b/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.223423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac37062-b0fe-473e-8ec6-aa6dd993b47b admin admin] RBAC: Authorizing `identity:create_grant(project_id=362ec4487088496a8198a3cf71815256, user_id=11be37af210c45fc9ddebf7f28b265ef, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.226106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac37062-b0fe-473e-8ec6-aa6dd993b47b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.231864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f19ad3068694411f9808277a34994ea2, user_id=6fbb043908dd4807b55c4c063105c26b, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.231864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.315124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 812/1604] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:25 2022] PUT /identity/v3/projects/362ec4487088496a8198a3cf71815256/users/11be37af210c45fc9ddebf7f28b265ef/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:25.316484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1644-3f22-44ee-b7fa-934863f54ed3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.316775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1644-3f22-44ee-b7fa-934863f54ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.317047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a1644-3f22-44ee-b7fa-934863f54ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.335175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9a0a1644-3f22-44ee-b7fa-934863f54ed3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:25.336429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9a0a1644-3f22-44ee-b7fa-934863f54ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:25.350709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 793/1605] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:25 2022] PUT /identity/v3/projects/f19ad3068694411f9808277a34994ea2/users/6fbb043908dd4807b55c4c063105c26b/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:25.351870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.495615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 813/1606] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:25.495615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ec1b-2b77-4d16-be67-8419debae0bb None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.495615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ec1b-2b77-4d16-be67-8419debae0bb None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.495615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ec1b-2b77-4d16-be67-8419debae0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.496444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c8259-1ff7-42f6-a9f8-15466067c2a4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.496444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.496444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.496444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.496962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.497059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-61a6ec1b-2b77-4d16-be67-8419debae0bb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:25.497059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-61a6ec1b-2b77-4d16-be67-8419debae0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:25.497059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:25.528669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.529484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 814/1607] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:25.531265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41114887-9f17-4b3f-9c7c-8b5fa5d54afb None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.531551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41114887-9f17-4b3f-9c7c-8b5fa5d54afb None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.531817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41114887-9f17-4b3f-9c7c-8b5fa5d54afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.534541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.546920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 794/1608] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:25.548605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.558643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3668f1-9ead-49e9-90ba-61f534243453 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.563597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.564364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.564642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.565120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] PATH_INFO: `/v3/projects/cd777f2fa3b149d5850a5af1301d319a/users/6f5aa83abd114e95baa9b04d4c5c6f43/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.568493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:25.574854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-41114887-9f17-4b3f-9c7c-8b5fa5d54afb None None] MFA Rules not processed for user `c735731e1e3c4fbca8fab3960dd4b4c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:25.576209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-41114887-9f17-4b3f-9c7c-8b5fa5d54afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:25.582199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd777f2fa3b149d5850a5af1301d319a, user_id=6f5aa83abd114e95baa9b04d4c5c6f43, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.585030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.641812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 815/1609] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:25.642966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a63a31-8bb7-49a8-a6a1-bfbc65994337 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.643242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a63a31-8bb7-49a8-a6a1-bfbc65994337 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.643507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a63a31-8bb7-49a8-a6a1-bfbc65994337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.663987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e4a63a31-8bb7-49a8-a6a1-bfbc65994337 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:25.665208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e4a63a31-8bb7-49a8-a6a1-bfbc65994337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:25.681151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 816/1610] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:25.682412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.682697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.682968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.705967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 795/1611] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:25 2022] PUT /identity/v3/projects/cd777f2fa3b149d5850a5af1301d319a/users/6f5aa83abd114e95baa9b04d4c5c6f43/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:25.706934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.724982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] MFA Rules not processed for user `6fbb043908dd4807b55c4c063105c26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:25.726599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:25.790868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858993e0-dde1-4e93-9960-b19eca9c4469 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.799980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69411bd2-5e7e-4280-b692-4b5eef612686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.800701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69411bd2-5e7e-4280-b692-4b5eef612686 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.800975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69411bd2-5e7e-4280-b692-4b5eef612686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.801159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69411bd2-5e7e-4280-b692-4b5eef612686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.807714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 817/1612] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:25.808675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.823963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b152be-74ed-4b7a-af59-51a8a133ea60 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.829615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213c23c-6681-4a4e-a94d-a96ee8f4b3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.835141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c23c-6681-4a4e-a94d-a96ee8f4b3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.835141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c23c-6681-4a4e-a94d-a96ee8f4b3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.835141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213c23c-6681-4a4e-a94d-a96ee8f4b3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.867117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:25.900163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69411bd2-5e7e-4280-b692-4b5eef612686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.901586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69411bd2-5e7e-4280-b692-4b5eef612686 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.905543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213c23c-6681-4a4e-a94d-a96ee8f4b3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.907078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213c23c-6681-4a4e-a94d-a96ee8f4b3e6 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.912225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 796/1613] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:25.913470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ea230-eaf6-4952-ac16-180bf3e977ad None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.913747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ea230-eaf6-4952-ac16-180bf3e977ad None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.914016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9ea230-eaf6-4952-ac16-180bf3e977ad None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.915355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 797/1614] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 10 11:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:25.915967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ea230-eaf6-4952-ac16-180bf3e977ad None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:25.919547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 818/1615] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:25.919547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d950f-bb7e-4d89-96b1-267b2b93ef1c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.919547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d950f-bb7e-4d89-96b1-267b2b93ef1c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.919547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d950f-bb7e-4d89-96b1-267b2b93ef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.928586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9ea230-eaf6-4952-ac16-180bf3e977ad None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:25.932238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c8fc6c-c5d1-49db-9735-56a6f39a3361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:25.932803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8fc6c-c5d1-49db-9735-56a6f39a3361 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.933076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8fc6c-c5d1-49db-9735-56a6f39a3361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.933349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c8fc6c-c5d1-49db-9735-56a6f39a3361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.949442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4c6d950f-bb7e-4d89-96b1-267b2b93ef1c None None] MFA Rules not processed for user `6f5aa83abd114e95baa9b04d4c5c6f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:25.949815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4c6d950f-bb7e-4d89-96b1-267b2b93ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:25.954607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8fc6c-c5d1-49db-9735-56a6f39a3361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:25.959790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c8fc6c-c5d1-49db-9735-56a6f39a3361 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:25.974210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 798/1616] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:25.975966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2c04-d561-4d31-8e51-9c0e84ed5f41 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.976275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2c04-d561-4d31-8e51-9c0e84ed5f41 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.976983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d2c04-d561-4d31-8e51-9c0e84ed5f41 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:25.977998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 799/1617] 10.222.0.13 () {54 vars in 962 bytes} [Wed Aug 10 11:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:25.987450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:25.988034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:25.988319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.043386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 819/1618] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:26.045128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.046018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.046803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.053169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:26.054827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:26.070877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:26.072212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:26.143649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 800/1619] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 10 11:22:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:26.144956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.161591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cbc00-eac2-4f2e-b677-4419057f178e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.167508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.168054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 820/1620] 10.222.0.13 () {58 vars in 1059 bytes} [Wed Aug 10 11:22:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:26.168582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.168959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.169170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.169525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.170221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:26.173542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:26.181522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c71020-9aac-4ab3-ae06-deacfc6df833 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.189343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.189956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.190231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.190654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.250665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:26.252489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:26.262024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 821/1621] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:26.263065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.278355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb6b70-7e57-474c-8d5d-7eeff677ea22 service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.298827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.299094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:26.304313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.304617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.304891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.313507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 801/1622] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:26 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:26.318774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.334979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:26.338107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:26.370827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 822/1623] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:26.440198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf0d2d4-8cce-44ed-9f88-0d212db5702a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.445970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a39c48-6c7f-419f-aa87-b1479131a35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.447032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a39c48-6c7f-419f-aa87-b1479131a35f admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.447318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a39c48-6c7f-419f-aa87-b1479131a35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.447834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a39c48-6c7f-419f-aa87-b1479131a35f admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.448529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a39c48-6c7f-419f-aa87-b1479131a35f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:26.450611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a39c48-6c7f-419f-aa87-b1479131a35f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:26.575828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 802/1624] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:26 2022] POST /identity/v3/users => generated 300 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:26.578150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.593968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4303509-a117-4ca0-a24a-e9668059aaca service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.616559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.617391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.617508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc4a5-68a6-480d-8b1f-aefd6f47e075 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.617702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc4a5-68a6-480d-8b1f-aefd6f47e075 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.617976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc4a5-68a6-480d-8b1f-aefd6f47e075 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.618284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.618690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.619640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 803/1625] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 10 11:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:26.624902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:26.628219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678188a-2148-405d-89d8-236e0b6fba65 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.628557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678188a-2148-405d-89d8-236e0b6fba65 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.628843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678188a-2148-405d-89d8-236e0b6fba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.630623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:26.648951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 823/1626] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:26.649979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.665572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b6ce7-bd10-440a-88dd-1507a135aff8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.675081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c678188a-2148-405d-89d8-236e0b6fba65 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:26.675081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c678188a-2148-405d-89d8-236e0b6fba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:26.677645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.679531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.680044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.680568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] PATH_INFO: `/v3/projects/e6878c358e5f4aadbf6d217e89f7b93d/users/18972a4499de4980ad4bd48ab1629c92/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.701483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6878c358e5f4aadbf6d217e89f7b93d, user_id=18972a4499de4980ad4bd48ab1629c92, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:26.707034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:26.749712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:26.815524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 804/1627] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:22:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:26.818302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440eaa34-dabc-46b7-9eee-1f6341851dea None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.818665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440eaa34-dabc-46b7-9eee-1f6341851dea None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.819061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440eaa34-dabc-46b7-9eee-1f6341851dea None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.824361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 805/1628] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 10 11:22:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:26.836150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.836521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.836862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.845192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 824/1629] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:26 2022] PUT /identity/v3/projects/e6878c358e5f4aadbf6d217e89f7b93d/users/18972a4499de4980ad4bd48ab1629c92/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:26.846241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:26.869797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:26.871592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:26.959481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc2667-1e8d-4825-90e2-f5b6e1e02449 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:26.969158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:26.970951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:26.971825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:26.972345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:26.980230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 806/1630] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:22:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:26.981740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:27.000515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:27.003967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:27.051639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 825/1631] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:27.053825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:27.079714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac8847-d57a-461c-b5ec-12ad8146e6d5 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:27.093583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:27.096629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:27.097095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:27.097886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:27.126182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:27.127963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:27.137343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eab934-a1e6-4e03-8d06-ac18d75438d1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:27.144522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 826/1632] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:27.152836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8998d-708d-48c3-a56b-63588ce7a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:27.153655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8998d-708d-48c3-a56b-63588ce7a512 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:27.154092 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8998d-708d-48c3-a56b-63588ce7a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:27.154361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8998d-708d-48c3-a56b-63588ce7a512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:27.155220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8998d-708d-48c3-a56b-63588ce7a512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:27.161559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8998d-708d-48c3-a56b-63588ce7a512 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:27.181211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 807/1633] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:27.188027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:27.205762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6d737-3b60-4b5b-ac2e-c03fa98109b6 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:27.228612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:27.231217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:27.231217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:27.232093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] PATH_INFO: `/v3/projects/e6878c358e5f4aadbf6d217e89f7b93d/users/18972a4499de4980ad4bd48ab1629c92/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:27.238273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6878c358e5f4aadbf6d217e89f7b93d, user_id=18972a4499de4980ad4bd48ab1629c92, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:27.243512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:27.294817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 827/1634] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:27 2022] PUT /identity/v3/projects/e6878c358e5f4aadbf6d217e89f7b93d/users/18972a4499de4980ad4bd48ab1629c92/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:27.300222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:27.300753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:27.301742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:27.357302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] MFA Rules not processed for user `18972a4499de4980ad4bd48ab1629c92`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:27.361496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:27.525514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 808/1635] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:27.843919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:28.376704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:28.471134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a547fec-a404-4ddf-84d9-427c15cb6964 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:28.476893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b5978a-a6de-4325-bc34-55193a1a17d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:28.477850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5978a-a6de-4325-bc34-55193a1a17d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:28.478469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5978a-a6de-4325-bc34-55193a1a17d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:28.478783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5978a-a6de-4325-bc34-55193a1a17d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:28.498119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5978a-a6de-4325-bc34-55193a1a17d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:28.498119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b5978a-a6de-4325-bc34-55193a1a17d9 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:28.521781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 828/1636] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:28 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:28.808350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:28.966732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743ea11-d46e-4d6d-9ed4-5d82658186fe None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:28.973652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:28.976517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:28.976517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:28.976517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] PATH_INFO: `/v3/users/6f5aa83abd114e95baa9b04d4c5c6f43` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:28.978319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5aa83abd114e95baa9b04d4c5c6f43)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:28.980899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:29.416513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5aa83abd114e95baa9b04d4c5c6f43'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:29.417687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5aa83abd114e95baa9b04d4c5c6f43'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:29.418653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5aa83abd114e95baa9b04d4c5c6f43'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:29.429585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5aa83abd114e95baa9b04d4c5c6f43'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:29.540195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5aa83abd114e95baa9b04d4c5c6f43'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:29.574057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 809/1637] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:28 2022] DELETE /identity/v3/users/6f5aa83abd114e95baa9b04d4c5c6f43 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:29.578436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:29.578963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:29.580130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:29.607463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:29.609549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:29.618184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:29.743508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:29.783898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 829/1638] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:29.857534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:29.860491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:30.057864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2788bb-9c9d-46b1-8a13-b75ff2f8ce62 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:30.076374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:30.077751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:30.078887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:30.079264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:30.113334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:30.114990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:30.133555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 810/1639] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:32.036792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:32.044121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:32.056281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8a9de-0024-49f6-a0c2-738f35e395c2 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:32.070409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:32.071806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:32.076058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:32.076814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] PATH_INFO: `/v3/projects/cd777f2fa3b149d5850a5af1301d319a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:32.078149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd777f2fa3b149d5850a5af1301d319a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:32.080312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:32.331915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd777f2fa3b149d5850a5af1301d319a', 'request_id': 'req-8767e5fd-1795-4179-ad62-13a7c2b87da1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:32.360217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd777f2fa3b149d5850a5af1301d319a', 'request_id': 'req-8767e5fd-1795-4179-ad62-13a7c2b87da1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:32.368451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd777f2fa3b149d5850a5af1301d319a', 'request_id': 'req-8767e5fd-1795-4179-ad62-13a7c2b87da1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:32.525530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] The token cache is being invalidate because project cd777f2fa3b149d5850a5af1301d319a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:32.525950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:32.527251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:32.532728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 830/1640] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:32 2022] DELETE /identity/v3/projects/cd777f2fa3b149d5850a5af1301d319a => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:32.533626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:32.574952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba752baf-0b95-4003-b492-25d61c7d5d69 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:32.704371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:32.705074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:32.705388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:32.705696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] PATH_INFO: `/v3/users/18972a4499de4980ad4bd48ab1629c92` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:32.707721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18972a4499de4980ad4bd48ab1629c92)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:32.709711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:32.894861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:32.895988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:32.919655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8767e5fd-1795-4179-ad62-13a7c2b87da1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:32.937260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:32.937260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:32.937260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:32.937260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] PATH_INFO: `/v3/users/c735731e1e3c4fbca8fab3960dd4b4c8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:32.940879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c735731e1e3c4fbca8fab3960dd4b4c8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:32.942243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: WARNING py.warnings [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:22:32.942243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: warnings.warn(msg) Aug 10 11:22:32.942243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: Aug 10 11:22:32.949240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:32.965643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:33.001534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18972a4499de4980ad4bd48ab1629c92'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.003315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18972a4499de4980ad4bd48ab1629c92'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.006132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18972a4499de4980ad4bd48ab1629c92'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.031602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18972a4499de4980ad4bd48ab1629c92'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.139708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-501c5bb1-f4f0-476c-a96e-29d99222da82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18972a4499de4980ad4bd48ab1629c92'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.187851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 811/1641] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:32 2022] DELETE /identity/v3/users/18972a4499de4980ad4bd48ab1629c92 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:22:33.310413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c735731e1e3c4fbca8fab3960dd4b4c8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.314361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c735731e1e3c4fbca8fab3960dd4b4c8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.315449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c735731e1e3c4fbca8fab3960dd4b4c8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.346552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c735731e1e3c4fbca8fab3960dd4b4c8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.454063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c735731e1e3c4fbca8fab3960dd4b4c8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:33.509692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 831/1642] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:32 2022] DELETE /identity/v3/users/c735731e1e3c4fbca8fab3960dd4b4c8 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:22:33.515445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5c12-96ef-4b44-8faa-1b0287727df1 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:33.515839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5c12-96ef-4b44-8faa-1b0287727df1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:33.518770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5c12-96ef-4b44-8faa-1b0287727df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:33.568896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f77a5c12-96ef-4b44-8faa-1b0287727df1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:33.571097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f77a5c12-96ef-4b44-8faa-1b0287727df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:33.696143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 812/1643] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:33.709514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:33.711028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:33.738452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:33.885972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429312ee-63f1-4233-992e-8ef1bf1b5bbb admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:33.890733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:33.891384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:33.891658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:33.891930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:33.914115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:33.919381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:33.940670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 832/1644] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:34.396249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ff7b-d7c7-428f-9cfd-06be3987cd3d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:34.396958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ff7b-d7c7-428f-9cfd-06be3987cd3d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:34.397308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27ff7b-d7c7-428f-9cfd-06be3987cd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:34.453866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-fd27ff7b-d7c7-428f-9cfd-06be3987cd3d None None] MFA Rules not processed for user `2386a816917a4c3d97f394b07d901386`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:34.455129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fd27ff7b-d7c7-428f-9cfd-06be3987cd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:34.615682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:34.620164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:34.655349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 813/1645] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:34.655752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39831555-c4e1-4ee3-afc2-3de59156aa96 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:34.656960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:34.657535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:34.658078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:34.668946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:34.669780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:34.669969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:34.671741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] PATH_INFO: `/v3/projects/e6878c358e5f4aadbf6d217e89f7b93d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:34.673324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6878c358e5f4aadbf6d217e89f7b93d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:34.675100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:34.702835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] MFA Rules not processed for user `2386a816917a4c3d97f394b07d901386`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:34.704661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:34.734965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 814/1646] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:34.745697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:34.747365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:34.875622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6878c358e5f4aadbf6d217e89f7b93d', 'request_id': 'req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:34.883548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:34.905901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6878c358e5f4aadbf6d217e89f7b93d', 'request_id': 'req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:34.915591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6878c358e5f4aadbf6d217e89f7b93d', 'request_id': 'req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:34.956966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e65e30-5593-4732-a598-9e7fcf2e7aab None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:34.995858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] The token cache is being invalidate because project e6878c358e5f4aadbf6d217e89f7b93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:34.996660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:34.999363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a98ed6d6-739a-4bd8-9e26-e9fe5630f69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:34.999855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 833/1647] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:34 2022] DELETE /identity/v3/projects/e6878c358e5f4aadbf6d217e89f7b93d => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:22:35.005709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc7798-fc4f-49bf-8a7b-78ad968aad3b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.006189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc7798-fc4f-49bf-8a7b-78ad968aad3b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.006811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc7798-fc4f-49bf-8a7b-78ad968aad3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.063612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d7bc7798-fc4f-49bf-8a7b-78ad968aad3b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:35.063612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d7bc7798-fc4f-49bf-8a7b-78ad968aad3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:35.105925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.106573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.109414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.109414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.109414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:35.184641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 834/1648] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:35.187090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.187587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.188134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.237799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.244296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:35.253068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:35.254620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:35.269162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 835/1649] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:35.276252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:35.282030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 815/1650] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:22:35.287636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57333ae5-41f2-446b-9241-01a0d692d494 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:35.298184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.299110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.300885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.301388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.302254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:35.304112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.306241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:35.325840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:35.326453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 836/1651] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:35.341147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b3918-9e17-4cf4-bed0-338720568742 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:35.348768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.349738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.350033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.350307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.350943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.353550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:35.516276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 816/1652] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:35 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:22:35.517640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:35.530543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99d331-dff2-460d-be10-22c99d49d17b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:35.543299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:35.606779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.607944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.608582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.609120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.610039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.612314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:35.711220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 837/1653] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:35 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:35.714795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:35.785467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aaaac1-db68-467a-b626-0bab14ffad3b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:35.785467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.785467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.785467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.788286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.788286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.788286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:35.806501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 817/1654] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:35.811398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:35.824899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42bd064-ced5-4fe8-95ef-fb592474cb40 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:35.839191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.839825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.840384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.840658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.841309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.843515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:35.928153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 838/1655] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:35.929029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:35.941108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df515d7b-a8f7-4483-a528-298ad37ab7e5 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:35.959076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b05b51-a9a2-4150-ad9b-96becec8d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:35.960389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05b51-a9a2-4150-ad9b-96becec8d06d admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:35.960903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05b51-a9a2-4150-ad9b-96becec8d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:35.961334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b05b51-a9a2-4150-ad9b-96becec8d06d admin admin] PATH_INFO: `/v3/projects/e71aab30a6544b65a5b89e9b5193e3f2/users/273301bd0b1d41f29ef60ed274fb7e99/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:35.994652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b05b51-a9a2-4150-ad9b-96becec8d06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e71aab30a6544b65a5b89e9b5193e3f2, user_id=273301bd0b1d41f29ef60ed274fb7e99, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:35.998088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b05b51-a9a2-4150-ad9b-96becec8d06d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:36.045204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:36.047790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:36.069655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dced0e5-aadb-417a-b05b-34e82985b715 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:36.099058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:36.099742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.100048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.100332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] PATH_INFO: `/v3/projects/b50f3ae6a06943ee9cce4bb812cfa5d1` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.101991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b50f3ae6a06943ee9cce4bb812cfa5d1)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:36.103980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:36.166083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 818/1656] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:35 2022] PUT /identity/v3/projects/e71aab30a6544b65a5b89e9b5193e3f2/users/273301bd0b1d41f29ef60ed274fb7e99/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:36.179912 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.180248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.180498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.259476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] MFA Rules not processed for user `273301bd0b1d41f29ef60ed274fb7e99`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:36.267718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:36.269261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:36.337533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b50f3ae6a06943ee9cce4bb812cfa5d1', 'request_id': 'req-e2857ec5-505e-407d-bdbe-11b640e3d997'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:36.356231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b50f3ae6a06943ee9cce4bb812cfa5d1', 'request_id': 'req-e2857ec5-505e-407d-bdbe-11b640e3d997'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:36.378109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b50f3ae6a06943ee9cce4bb812cfa5d1', 'request_id': 'req-e2857ec5-505e-407d-bdbe-11b640e3d997'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:36.476353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] The token cache is being invalidate because project b50f3ae6a06943ee9cce4bb812cfa5d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:36.477449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:36.478757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e2857ec5-505e-407d-bdbe-11b640e3d997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:36.482158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 839/1657] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:36 2022] DELETE /identity/v3/projects/b50f3ae6a06943ee9cce4bb812cfa5d1 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:22:36.487799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c5ff4-aaec-47dd-a1d4-91c88449c3e4 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.488123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c5ff4-aaec-47dd-a1d4-91c88449c3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.488407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1c5ff4-aaec-47dd-a1d4-91c88449c3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.523890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 819/1658] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:36.530013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:36.544277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fc1c5ff4-aaec-47dd-a1d4-91c88449c3e4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:36.546074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fc1c5ff4-aaec-47dd-a1d4-91c88449c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:36.722889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 840/1659] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:36.723716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30be04-b4c1-41de-9ae2-d256e5aa1f56 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:36.724751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.725041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.725318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.734649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b2541f-c859-4d38-b8ad-dbe0c0ad753f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:36.736277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2541f-c859-4d38-b8ad-dbe0c0ad753f admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.747199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2541f-c859-4d38-b8ad-dbe0c0ad753f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.747199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b2541f-c859-4d38-b8ad-dbe0c0ad753f admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.747199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2541f-c859-4d38-b8ad-dbe0c0ad753f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:36.747199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b2541f-c859-4d38-b8ad-dbe0c0ad753f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:36.761333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:36.762590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:36.784782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 841/1660] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:36.789523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:36.815299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2291169-d5d5-454a-a17b-5ef25eff412d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:36.819535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:36.820719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.821201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.821602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.825960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:36.826639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:36.828650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cb2787-9ed2-439d-b8a5-f3a6c933b21d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:36.865101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 842/1661] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:36.867110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6d08-685c-4dc2-b975-cddb43b269e2 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.867632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6d08-685c-4dc2-b975-cddb43b269e2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.868159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615a6d08-685c-4dc2-b975-cddb43b269e2 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.869902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 843/1662] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 10 11:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:36.871757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6d08-685c-4dc2-b975-cddb43b269e2 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:36.886429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 820/1663] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:36 2022] POST /identity/v3/projects => generated 381 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:36.889927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615a6d08-685c-4dc2-b975-cddb43b269e2 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:36.891548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:36.892302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:36.893210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:36.996255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:36.999083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:37.002718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:37.033070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.036222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.038718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.040809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.045071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.047459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.137065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 821/1664] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:22:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:37.138261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.140682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:37.180445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 844/1665] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:36 2022] POST /identity/v3/projects => generated 405 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:37.181344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.196798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea3bf2a-c54e-4d85-bbc5-d010060c00c0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.262856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:37.307488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2b46d-3138-4cd0-83a4-732f49547910 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.311892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.312809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.313090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.313361 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.314203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.316409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.343489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.343979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.344398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.344673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.345680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.347277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.427196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 822/1666] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:37 2022] POST /identity/v3/users => generated 297 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:37.431306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.451028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3c64d5-7bdc-4544-8539-618725e4574e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.461638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.463773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.463773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.463773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.464600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.466455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.480303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 823/1667] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:37.491661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 845/1668] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:37 2022] POST /identity/v3/users => generated 303 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:37.495596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.496924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.507879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e07f80-dc7c-4a52-819c-0afb9ae8d1da admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.519579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b0b34-433c-404a-88f6-cee52d5db85e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.521025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.523917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.524501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.524968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] PATH_INFO: `/v3/projects/168eeee23e5f4586b0df5db55658369d/users/872961c2a9374a719ea28b850a80b0be/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.542806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.543450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.543727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.543989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.544762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.547068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.553318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168eeee23e5f4586b0df5db55658369d, user_id=872961c2a9374a719ea28b850a80b0be, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.562927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.590005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 846/1669] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:37.590924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.598056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:37.648804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 824/1670] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:37 2022] PUT /identity/v3/projects/168eeee23e5f4586b0df5db55658369d/users/872961c2a9374a719ea28b850a80b0be/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:37.649808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.739178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d009d-752c-48ad-95e3-48ddd6c1c239 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.796951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016b052-9737-4b6f-abd3-b18258803fe4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.809821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.810749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.811061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.811424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.812110 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.813750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.839887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 825/1671] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:37.845564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.846190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.846463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.846892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.848073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.865002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.870506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.875051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150155d5-36e5-45f7-a3ec-9ce675c5fa19 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.881888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.882985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.882985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.883193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.883740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.885373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.892440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 847/1672] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:37.893420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.908108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 826/1673] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:37.914862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:37.921832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed5195-18ff-430b-84f8-c30c0dbbdd21 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.935332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b193b51-a8a1-473f-bcbc-f0126695e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.939150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b193b51-a8a1-473f-bcbc-f0126695e9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.939447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b193b51-a8a1-473f-bcbc-f0126695e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.939913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b193b51-a8a1-473f-bcbc-f0126695e9ae admin admin] PATH_INFO: `/v3/projects/855fec46478349ea978f0174c643ff06/users/3e48377440f445e2b55682bd1161811c/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.950782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfd686-ab1b-4ac8-a96f-651d5a3729ce admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:37.956069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f26c78-a693-4a97-afaa-d9ee3f609f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:37.956756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f26c78-a693-4a97-afaa-d9ee3f609f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:37.957040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f26c78-a693-4a97-afaa-d9ee3f609f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:37.957315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f26c78-a693-4a97-afaa-d9ee3f609f54 admin admin] PATH_INFO: `/v3/projects/168eeee23e5f4586b0df5db55658369d/users/872961c2a9374a719ea28b850a80b0be/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:37.959425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f26c78-a693-4a97-afaa-d9ee3f609f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=168eeee23e5f4586b0df5db55658369d, user_id=872961c2a9374a719ea28b850a80b0be, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.962442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f26c78-a693-4a97-afaa-d9ee3f609f54 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:37.976977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b193b51-a8a1-473f-bcbc-f0126695e9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=855fec46478349ea978f0174c643ff06, user_id=3e48377440f445e2b55682bd1161811c, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:37.988078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b193b51-a8a1-473f-bcbc-f0126695e9ae admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:38.000037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 827/1674] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:37 2022] PUT /identity/v3/projects/168eeee23e5f4586b0df5db55658369d/users/872961c2a9374a719ea28b850a80b0be/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:38.001929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:38.001929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:38.002123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:38.064291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] MFA Rules not processed for user `872961c2a9374a719ea28b850a80b0be`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:38.065661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:38.134712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 848/1675] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:37 2022] PUT /identity/v3/projects/855fec46478349ea978f0174c643ff06/users/3e48377440f445e2b55682bd1161811c/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:38.137946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:38.138231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:38.138506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:38.189825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:38.191280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:38.249466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 828/1676] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:38.258976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:38.261210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:38.263020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:38.351660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 849/1677] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:38.397220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ec681-5704-46e7-adcd-75f6d9d8df89 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:38.444857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:38.449529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:38.460591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:38.498052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf150c9-d49b-435e-8ca0-67a9a8dc73e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:38.499685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf150c9-d49b-435e-8ca0-67a9a8dc73e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:38.499685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf150c9-d49b-435e-8ca0-67a9a8dc73e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:38.499879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf150c9-d49b-435e-8ca0-67a9a8dc73e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:38.619964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e532-3c6b-4223-9373-4bdede179522 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:38.647417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c199f-07fe-4ac3-8566-c02618d18a63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:38.648077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c199f-07fe-4ac3-8566-c02618d18a63 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:38.648355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c199f-07fe-4ac3-8566-c02618d18a63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:38.648623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c199f-07fe-4ac3-8566-c02618d18a63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:38.690599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c199f-07fe-4ac3-8566-c02618d18a63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:38.703125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259c199f-07fe-4ac3-8566-c02618d18a63 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:38.757415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 850/1678] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:38.771585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf150c9-d49b-435e-8ca0-67a9a8dc73e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:38.775418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf150c9-d49b-435e-8ca0-67a9a8dc73e0 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:38.790322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 829/1679] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:22:39.509513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869c93-31bd-425e-8d3c-f63a198996e8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:39.509886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869c93-31bd-425e-8d3c-f63a198996e8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:39.510246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef869c93-31bd-425e-8d3c-f63a198996e8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:39.511952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 851/1680] 10.222.0.13 () {54 vars in 961 bytes} [Wed Aug 10 11:22:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:39.526746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:39.526746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:39.526746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:39.565064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:39.568106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:39.683572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:39.765077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 830/1681] 10.222.0.13 () {58 vars in 1058 bytes} [Wed Aug 10 11:22:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:39.769887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869c93-31bd-425e-8d3c-f63a198996e8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:39.793738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef869c93-31bd-425e-8d3c-f63a198996e8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:39.810408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:39.810408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:39.810408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:39.810408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:39.813117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:39.860300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:39.874628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:39.905162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 852/1682] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:40.329200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:40.346405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67bed4-6e07-46ed-9f0d-0fa98d0411a2 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:40.361427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872961c2a9374a719ea28b850a80b0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168eeee23e5f4586b0df5db55658369d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:40.362101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:40.362393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:40.362746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:40.363657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:40.363966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:40.368962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:40.394492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 831/1683] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:40.398279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:40.415645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c62a09-0264-4044-940a-3ef4edd28aa5 service placement] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:40.425185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872961c2a9374a719ea28b850a80b0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168eeee23e5f4586b0df5db55658369d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:40.425841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:40.426122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:40.426394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:40.427036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:40.428946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:40.543062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 853/1684] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:40 2022] POST /identity/v3/projects => generated 355 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:40.547507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:40.575449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68e2a3-baa0-4016-ad3b-465a4cd4a0f9 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:40.827802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872961c2a9374a719ea28b850a80b0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168eeee23e5f4586b0df5db55658369d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:40.828693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:40.829138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:40.831015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:40.832492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:40.834204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:40.838172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf443db-6cbe-44bb-bd4f-333763bb8ffe tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:40.884457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 832/1685] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:40.957608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:40.959894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:40.973871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ed25f1-75b4-452e-8de6-1c69448dbfe0 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:40.988413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '872961c2a9374a719ea28b850a80b0be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '168eeee23e5f4586b0df5db55658369d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:40.989233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:40.990568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:40.991220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] PATH_INFO: `/v3/projects/783b1f383c624ff09790dd97baa8b991` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:40.992671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorizing `identity:delete_project(project_id=783b1f383c624ff09790dd97baa8b991)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:40.997583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:41.205512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '783b1f383c624ff09790dd97baa8b991', 'request_id': 'req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:41.228148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '783b1f383c624ff09790dd97baa8b991', 'request_id': 'req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:41.252100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '783b1f383c624ff09790dd97baa8b991', 'request_id': 'req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:41.329848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:41.330353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:41.330906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:41.369596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] MFA Rules not processed for user `11be37af210c45fc9ddebf7f28b265ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:41.370814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:41.399802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] The token cache is being invalidate because project 783b1f383c624ff09790dd97baa8b991 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:41.400177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:41.402780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:41.407676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 854/1686] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:40 2022] DELETE /identity/v3/projects/783b1f383c624ff09790dd97baa8b991 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:41.486572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 833/1687] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:41.511145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:41.511145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:41.620997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc94f1-d43c-4c67-ad0d-22b9e1148270 tempest-VolumeQuotaClassesTest-1194798120 tempest-VolumeQuotaClassesTest-1194798120-project-admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:41.630379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de2ed2-e100-4a7c-8752-41d40a2d02d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:41.631293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de2ed2-e100-4a7c-8752-41d40a2d02d6 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:41.631690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de2ed2-e100-4a7c-8752-41d40a2d02d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:41.632172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de2ed2-e100-4a7c-8752-41d40a2d02d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:41.643647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de2ed2-e100-4a7c-8752-41d40a2d02d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:41.646255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77de2ed2-e100-4a7c-8752-41d40a2d02d6 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:41.682887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 855/1688] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:42.729495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:42.731145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:42.829326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:42.847915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0dc623-1466-404d-9a78-26f0cb26ff9e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:42.862383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:42.864206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:42.864629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:42.865580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] PATH_INFO: `/v3/users/273301bd0b1d41f29ef60ed274fb7e99` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:42.867555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] RBAC: Authorizing `identity:delete_user(user_id=273301bd0b1d41f29ef60ed274fb7e99)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:42.871279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:42.924733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:42.925607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:42.926227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:43.019290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] MFA Rules not processed for user `3e48377440f445e2b55682bd1161811c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:43.022031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:43.118650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '273301bd0b1d41f29ef60ed274fb7e99'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:43.120319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '273301bd0b1d41f29ef60ed274fb7e99'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:43.121631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '273301bd0b1d41f29ef60ed274fb7e99'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:43.134053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '273301bd0b1d41f29ef60ed274fb7e99'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:43.177274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:43.181590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 856/1689] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:42 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:43.187134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:43.193974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:43.213583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47895fe3-f5dc-4aa4-83a3-5bafcfd92f63 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:43.219813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:43.221028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:43.221477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:43.221886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:43.222740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:43.225005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d90b1e27-1b24-4ab9-a1ee-5429ba30466b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '273301bd0b1d41f29ef60ed274fb7e99'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:43.225797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:43.258834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 834/1690] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:42 2022] DELETE /identity/v3/users/273301bd0b1d41f29ef60ed274fb7e99 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:43.260314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e5c34-39ab-4264-a2bf-e079b79069ad None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:43.269707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e5c34-39ab-4264-a2bf-e079b79069ad None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:43.269707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e5c34-39ab-4264-a2bf-e079b79069ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:43.302245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0c4e5c34-39ab-4264-a2bf-e079b79069ad None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:43.304381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0c4e5c34-39ab-4264-a2bf-e079b79069ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:43.379504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 857/1691] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:43 2022] POST /identity/v3/projects => generated 405 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:43.385235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:43.554533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 835/1692] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:43.555859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:43.556181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:43.557381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:43.651076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] MFA Rules not processed for user `87adb637dd814cf8be9ff6c2a2bc26ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:43.652517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:43.679179 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980e0001-6be8-476c-8e14-ffb44b63e6f0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:43.693551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:43.694356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:43.695216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:43.695607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:43.696406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:43.698797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:43.817608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 858/1693] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:43 2022] POST /identity/v3/users => generated 309 bytes in 437 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:43.821639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:43.831640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:43.874111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:43.893930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 836/1694] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:43.896103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:43.917839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490aac1b-9d6e-4400-ba9e-92b97bf56109 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:43.928499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:43.929134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:43.929415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:43.929689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:43.930342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:43.931986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:43.966123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 837/1695] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:43.969292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:43.973105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:44.017960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59dcf23-76c3-456b-8278-49191c595a2b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:44.031282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:44.031929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.032232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.032502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.058767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:44.060453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:44.099255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 859/1696] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:44.111639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:44.138449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2371aac-d6b1-42ba-9457-27a3fcafdfa2 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:44.149707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:44.150402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.150677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.150945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] PATH_INFO: `/v3/projects/14b9b464fc4e410ebab1fdacc0da1286/users/4fe6d4cfa16146598e9cba54abed2733/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.187484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80135fa-1e80-46c5-9115-00d6806ab3b6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:44.209406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b9b464fc4e410ebab1fdacc0da1286, user_id=4fe6d4cfa16146598e9cba54abed2733, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:44.212134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:44.234799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:44.235744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.235955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.236260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.256409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:44.259132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:44.279531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 838/1697] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:44.289268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:44.291060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:44.311194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5e496-bef8-4ad5-8dd9-9193022b1d87 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:44.330829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:44.331918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.332053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.332479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] PATH_INFO: `/v3/users/6fbb043908dd4807b55c4c063105c26b` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.335370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbb043908dd4807b55c4c063105c26b)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:44.339744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:44.370344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 860/1698] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:44 2022] PUT /identity/v3/projects/14b9b464fc4e410ebab1fdacc0da1286/users/4fe6d4cfa16146598e9cba54abed2733/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:44.374942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:44.541902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8a6dd-a1ef-42ea-85c6-245814b7cfc7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:44.549177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:44.549809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.550089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.550360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.551008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:44.555097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:44.590643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 861/1699] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:44.595404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:44.606814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9e4fd-879e-49a4-ac40-678e046bd71f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:44.615538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72c9e-3f70-47ea-9a86-b46d1036a8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:44.616277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72c9e-3f70-47ea-9a86-b46d1036a8e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.616558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72c9e-3f70-47ea-9a86-b46d1036a8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.616889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72c9e-3f70-47ea-9a86-b46d1036a8e0 admin admin] PATH_INFO: `/v3/projects/14b9b464fc4e410ebab1fdacc0da1286/users/4fe6d4cfa16146598e9cba54abed2733/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.619082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c72c9e-3f70-47ea-9a86-b46d1036a8e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b9b464fc4e410ebab1fdacc0da1286, user_id=4fe6d4cfa16146598e9cba54abed2733, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:44.621967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c72c9e-3f70-47ea-9a86-b46d1036a8e0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:44.635310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 862/1700] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:44 2022] PUT /identity/v3/projects/14b9b464fc4e410ebab1fdacc0da1286/users/4fe6d4cfa16146598e9cba54abed2733/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:22:44.637034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbb043908dd4807b55c4c063105c26b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:44.638001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbb043908dd4807b55c4c063105c26b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:44.639550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbb043908dd4807b55c4c063105c26b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:44.653658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbb043908dd4807b55c4c063105c26b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:44.749789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbb043908dd4807b55c4c063105c26b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:44.785649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 839/1701] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:44 2022] DELETE /identity/v3/users/6fbb043908dd4807b55c4c063105c26b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:44.791062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2fd27-b587-4a50-a0ba-393367730c91 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:44.791360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2fd27-b587-4a50-a0ba-393367730c91 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:44.792279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2fd27-b587-4a50-a0ba-393367730c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:44.826164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-40f2fd27-b587-4a50-a0ba-393367730c91 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:44.828133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-40f2fd27-b587-4a50-a0ba-393367730c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:44.910961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:44.986280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 863/1702] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:45.107321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:45.110279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:45.227481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ceb62e-cf23-4fea-8d48-cd73d4f9ca21 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:45.245885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:45.247002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:45.247522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:45.248060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:45.308758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:45.308758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:45.313920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 840/1703] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:45.549494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:45.980826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:45.983196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:45.983740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:46.075513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:46.078159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:46.191910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 864/1704] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:22:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:46.753214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:46.755624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:46.965618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:46.965793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33686350-55aa-419b-9135-2a145d0a90d7 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:46.968155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:46.973274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdb516-8c02-4621-b2c1-716d4b6c5e4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:46.977632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb516-8c02-4621-b2c1-716d4b6c5e4c service swift] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:46.977632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb516-8c02-4621-b2c1-716d4b6c5e4c service swift] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:46.977632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdb516-8c02-4621-b2c1-716d4b6c5e4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:46.989366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdb516-8c02-4621-b2c1-716d4b6c5e4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:46.991298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdb516-8c02-4621-b2c1-716d4b6c5e4c service swift] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:46.997957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78590229-3295-4374-8ae4-bb3c53d9fb04 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:47.019859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:47.020818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:47.021270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:47.021695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] PATH_INFO: `/v3/projects/e71aab30a6544b65a5b89e9b5193e3f2` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:47.022927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 841/1705] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:22:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:22:47.027850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e71aab30a6544b65a5b89e9b5193e3f2)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:47.030090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:47.210117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:47.234867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e71aab30a6544b65a5b89e9b5193e3f2', 'request_id': 'req-88177273-4fd3-4308-b7a8-47c417f9ff69'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:47.256093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e71aab30a6544b65a5b89e9b5193e3f2', 'request_id': 'req-88177273-4fd3-4308-b7a8-47c417f9ff69'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:47.276181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e71aab30a6544b65a5b89e9b5193e3f2', 'request_id': 'req-88177273-4fd3-4308-b7a8-47c417f9ff69'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:47.302264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:47.302579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:47.302858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:47.408483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] MFA Rules not processed for user `0782e27e2318438fbf155dfa9cb834b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:47.413470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:47.483636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] The token cache is being invalidate because project e71aab30a6544b65a5b89e9b5193e3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:47.484123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:47.484898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88177273-4fd3-4308-b7a8-47c417f9ff69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:47.488346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 865/1706] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:46 2022] DELETE /identity/v3/projects/e71aab30a6544b65a5b89e9b5193e3f2 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:47.494779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:47.494779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:47.494779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:47.572327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] MFA Rules not processed for user `9b5890ee9a114299b0a15a6608503632`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:47.575019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:47.671734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 842/1707] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:47.672628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:47.682344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:47.754767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 866/1708] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:47.757940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:47.764895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:47.799283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:47.819507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f6cd85-9f42-446c-ad85-833b414f7928 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:47.828792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:47.829765 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:47.830238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:47.830668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] PATH_INFO: `/v3/users/872961c2a9374a719ea28b850a80b0be` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:47.832254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] RBAC: Authorizing `identity:delete_user(user_id=872961c2a9374a719ea28b850a80b0be)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:47.835074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:47.944953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973d748e-73db-4b85-9fcd-4082e84ef8ae None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:47.954489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0f2e7-67f0-466b-9bfe-af565135bd96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:47.955797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0f2e7-67f0-466b-9bfe-af565135bd96 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:47.956272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0f2e7-67f0-466b-9bfe-af565135bd96 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:47.956705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0f2e7-67f0-466b-9bfe-af565135bd96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:47.967886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0f2e7-67f0-466b-9bfe-af565135bd96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:47.969925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0f2e7-67f0-466b-9bfe-af565135bd96 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:48.040995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 867/1709] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:48.050246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:48.054183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:48.055890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:48.165414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872961c2a9374a719ea28b850a80b0be'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:48.166468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872961c2a9374a719ea28b850a80b0be'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:48.167283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872961c2a9374a719ea28b850a80b0be'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:48.184401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872961c2a9374a719ea28b850a80b0be'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:48.230059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] MFA Rules not processed for user `9b5890ee9a114299b0a15a6608503632`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:48.250233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:48.264105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872961c2a9374a719ea28b850a80b0be'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:48.328995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 868/1710] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:48.336058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 843/1711] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:47 2022] DELETE /identity/v3/users/872961c2a9374a719ea28b850a80b0be => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:48.348516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:48.354120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:48.456716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d6d47-10b1-46b0-af52-3d007b60b1f0 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:48.574393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:48.662340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:48.663267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:48.664386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:48.664386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:48.667111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:48.763529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:48.766326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:48.783719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 869/1712] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:49.723455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:49.725192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:49.936889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0654f40a-43a3-4156-94c1-df6d297d501c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:49.956951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] PATH_INFO: `/v3/projects/f19ad3068694411f9808277a34994ea2` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f19ad3068694411f9808277a34994ea2)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: WARNING py.warnings [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: warnings.warn(msg) Aug 10 11:22:49.961646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: Aug 10 11:22:49.962682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:50.599116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f19ad3068694411f9808277a34994ea2', 'request_id': 'req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:50.624051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:50.624051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:50.652286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f19ad3068694411f9808277a34994ea2', 'request_id': 'req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:50.653400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5ac53-8eb9-4b51-8059-eb176b1da831 service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:50.673967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f19ad3068694411f9808277a34994ea2', 'request_id': 'req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:50.776536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:50.787704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] The token cache is being invalidate because project f19ad3068694411f9808277a34994ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:50.788352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:50.789752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4fc01df4-4e59-4675-bdd0-d2d1715d88fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:50.801148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 844/1713] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:49 2022] DELETE /identity/v3/projects/f19ad3068694411f9808277a34994ea2 => generated 0 bytes in 1078 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:50.802495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:50.807091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839ba35-19e5-4777-9e80-c4d6227a9d37 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:50.807504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:50.807803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839ba35-19e5-4777-9e80-c4d6227a9d37 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:50.808173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:50.808537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839ba35-19e5-4777-9e80-c4d6227a9d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:50.808908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] PATH_INFO: `/v3/projects/168eeee23e5f4586b0df5db55658369d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:50.817411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=168eeee23e5f4586b0df5db55658369d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:50.822294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:50.894624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0839ba35-19e5-4777-9e80-c4d6227a9d37 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:50.900211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0839ba35-19e5-4777-9e80-c4d6227a9d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:50.953432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 845/1714] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:50.956948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:50.957316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:50.957703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.070199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:51.072207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:51.082833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '168eeee23e5f4586b0df5db55658369d', 'request_id': 'req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:51.103401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '168eeee23e5f4586b0df5db55658369d', 'request_id': 'req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:51.121586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '168eeee23e5f4586b0df5db55658369d', 'request_id': 'req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:51.258954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:51.264601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] The token cache is being invalidate because project 168eeee23e5f4586b0df5db55658369d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:22:51.264779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:51.265621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-86601ea9-4ce2-4e3c-8b10-fcb0c55a1c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:22:51.268261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 870/1715] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:22:50 2022] DELETE /identity/v3/projects/168eeee23e5f4586b0df5db55658369d => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:51.276214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e0af-62b0-4ab5-9dbf-d07bdb454d7d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.276760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e0af-62b0-4ab5-9dbf-d07bdb454d7d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.278180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d86e0af-62b0-4ab5-9dbf-d07bdb454d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.301693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 846/1716] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:51.305380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:51.316436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2d86e0af-62b0-4ab5-9dbf-d07bdb454d7d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:51.322372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2d86e0af-62b0-4ab5-9dbf-d07bdb454d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:51.333567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 871/1717] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:51.342892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.350578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.350578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.350924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9b99c2-9104-42e5-8ec7-9e298d298062 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:51.401227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:51.405076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.405076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.405076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.405076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:51.405701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:51.407000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:51.422882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:51.424859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:51.429128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 847/1718] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:51.444227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 872/1719] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:51.459195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:51.465936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:51.470790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:51.479708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607738c-01c0-4813-b02f-083a443c8742 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:51.485802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:51.486612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.487000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.487372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.488258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:22:51.488876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:51.499245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6d538-9c47-4264-a3dd-ddb6a21e0c35 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:51.502168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a13317-8f30-4a97-abb6-4f383f27e72b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:51.518752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:51.519464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.520683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.521056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.521805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:51.523814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:51.536032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 848/1720] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:22:51.563418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.563418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.563418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.619932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] MFA Rules not processed for user `4fe6d4cfa16146598e9cba54abed2733`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:51.624471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:51.814785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 849/1721] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:51.815707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:51.850470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9fcf1-7043-4448-8e97-8183a1bb3a30 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:51.866286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:51.872330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:51.872330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:51.872330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:51.872330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:51.872330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:51.899326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 873/1722] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:51 2022] POST /identity/v3/projects => generated 367 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:51.903947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.053525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93952fd-658d-49b3-be22-f170e859ac32 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.065978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24394596-0cd0-451b-93c0-877f0c124db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.066763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24394596-0cd0-451b-93c0-877f0c124db5 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.067772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24394596-0cd0-451b-93c0-877f0c124db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.070339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24394596-0cd0-451b-93c0-877f0c124db5 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.070999 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 850/1723] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:51 2022] POST /identity/v3/projects => generated 391 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:52.074311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24394596-0cd0-451b-93c0-877f0c124db5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.075037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.080198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24394596-0cd0-451b-93c0-877f0c124db5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.221127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 874/1724] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:51 2022] POST /identity/v3/users => generated 284 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:52.222779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd4682-4a9d-424d-9b5b-2e4ff17923f2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.223028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd4682-4a9d-424d-9b5b-2e4ff17923f2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.223430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd4682-4a9d-424d-9b5b-2e4ff17923f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.303071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0ae9d-09d6-4897-987f-1648246882dd admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.307471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2ccd4682-4a9d-424d-9b5b-2e4ff17923f2 None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:52.315967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2ccd4682-4a9d-424d-9b5b-2e4ff17923f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:52.317225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.317863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.318142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.318410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.320081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.322722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.479387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 851/1725] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:52 2022] POST /identity/v3/users => generated 296 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:22:52.480448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.493778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fe7525-29ca-40e5-87d0-80a94d733852 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.496313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 875/1726] 10.222.0.45 () {58 vars in 1015 bytes} [Wed Aug 10 11:22:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:52.496675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.497109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.497493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.499176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.499838 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.500197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.500474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.501118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.502823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.518557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 852/1727] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:52.519813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.531955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65697f-021e-4e29-898c-ad4a919aea69 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.536991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] MFA Rules not processed for user `9b5890ee9a114299b0a15a6608503632`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:52.540743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:52.542986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.544254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.544554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.544842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.548611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.552600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.583514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 853/1728] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:52.584780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.600200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494a7f89-488c-4a33-a4b2-b329e88c8467 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.617261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.623709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.623709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.623709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.623709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.623709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.645713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 854/1729] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:52.651351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.656631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:52.667562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 876/1730] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:52.668552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.694118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14793af5-8b08-4301-ac77-324d44de33ac None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.706526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.707257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.707602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c9509-bb74-4ece-883b-62a9d515c038 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.707725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.708156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.708925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.710646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.728173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.728537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.728884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.729159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.734311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 877/1731] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:52.735252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.754772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.756360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.758515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3e68-f6f3-4344-9753-f924b82d421d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.772860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcd09e-cd9e-4a30-9805-021a36a3fdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.773889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcd09e-cd9e-4a30-9805-021a36a3fdf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.774318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcd09e-cd9e-4a30-9805-021a36a3fdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.774736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcd09e-cd9e-4a30-9805-021a36a3fdf5 admin admin] PATH_INFO: `/v3/projects/a063e909b4544557993de6d039b22eee/users/a5c003e4470a46ce90f3f01b03ed5766/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.775894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 855/1732] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:52.776850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:52.807750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ce1e5-a114-4e27-9737-93e6dc86ebb0 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:52.820149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcd09e-cd9e-4a30-9805-021a36a3fdf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a063e909b4544557993de6d039b22eee, user_id=a5c003e4470a46ce90f3f01b03ed5766, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.841523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcd09e-cd9e-4a30-9805-021a36a3fdf5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:52.842578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e965cfb3-843a-423d-b468-b68cbf8d9090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:52.842578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965cfb3-843a-423d-b468-b68cbf8d9090 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:52.842578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965cfb3-843a-423d-b468-b68cbf8d9090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:52.858204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e965cfb3-843a-423d-b468-b68cbf8d9090 admin admin] PATH_INFO: `/v3/projects/6e06f66aefac4b0f8a5a6df17a217888/users/f44bf26e27c546d085a0ed7cf102f105/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:52.912811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965cfb3-843a-423d-b468-b68cbf8d9090 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e06f66aefac4b0f8a5a6df17a217888, user_id=f44bf26e27c546d085a0ed7cf102f105, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:52.912811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e965cfb3-843a-423d-b468-b68cbf8d9090 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:53.242226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 878/1733] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:52 2022] PUT /identity/v3/projects/a063e909b4544557993de6d039b22eee/users/a5c003e4470a46ce90f3f01b03ed5766/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:53.242805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 856/1734] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:52 2022] PUT /identity/v3/projects/6e06f66aefac4b0f8a5a6df17a217888/users/f44bf26e27c546d085a0ed7cf102f105/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:53.255565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:53.256702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:53.257153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:53.271906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:53.272231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:53.272504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:53.383559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] MFA Rules not processed for user `f44bf26e27c546d085a0ed7cf102f105`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:53.387669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] MFA Rules not processed for user `a5c003e4470a46ce90f3f01b03ed5766`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:53.388815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:53.391001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:53.600423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 857/1735] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:53.604151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:53.604603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 879/1736] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:53 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:53.616173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:53.709409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:53.799681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec404e9c-71de-455a-ad03-f2ea005b8100 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:53.813499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034b269-39d0-4494-9eea-56810fedebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:53.814139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034b269-39d0-4494-9eea-56810fedebda admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:53.814428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034b269-39d0-4494-9eea-56810fedebda admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:53.814699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034b269-39d0-4494-9eea-56810fedebda admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:53.822026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034b269-39d0-4494-9eea-56810fedebda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:53.824486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034b269-39d0-4494-9eea-56810fedebda admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:53.848912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce7be4-7078-4fcc-ae73-b5fb1e8d053e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:53.848912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:53.866753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:53.867083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:53.867365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:53.868895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:53.875258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:54.013772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 858/1737] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:53 2022] POST /identity/v3/projects => generated 367 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:54.015885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.016350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.016949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.049123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] MFA Rules not processed for user `11be37af210c45fc9ddebf7f28b265ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:54.050704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:54.059415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 880/1738] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:22:53 2022] POST /identity/v3/projects => generated 393 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:54.059415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:54.165258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 859/1739] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:54.167618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:54.213798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:54.259198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1805903-99ea-4d9f-9e41-63dc2ea332fa admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:54.275876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:54.276635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.277003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.277340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.278046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:54.280944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:54.344918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a137a1f-cb5c-4d44-a65b-737779e9f606 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:54.354881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:54.356055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.358662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.359347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.360356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:54.363384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:54.611260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 881/1740] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:54 2022] POST /identity/v3/users => generated 290 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:54.614471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:54.631172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 860/1741] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:22:54 2022] POST /identity/v3/users => generated 303 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:22:54.635119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:54.659469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b61188e-dd61-4cdf-856a-0e6a2ddbdc8b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:54.679203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:54.684722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.684722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.684722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.684722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:54.684722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:54.714362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad987c4b-6b15-4a24-a70c-f3520741ac88 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:54.738160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:54.741826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.742983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.743884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.752157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:54.758771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:54.762633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 882/1742] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:54.764487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:54.801027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabbed00-2bed-457e-a48b-1cb3bdc354d2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:54.804390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 861/1743] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:54 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:22:54.808296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:54.816919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:54.816919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.816919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.829368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] PATH_INFO: `/v3/projects/f1d7149a3ef44255a69bfb65bca651cd/users/42200db3f7e74c36bcbecbdd741052ec/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.838042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1897221-2392-4337-a3ab-fb762908e40f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:54.845185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc216636-a5a5-42c9-8f95-2b30c4f3c5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:54.847231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216636-a5a5-42c9-8f95-2b30c4f3c5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:54.847690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216636-a5a5-42c9-8f95-2b30c4f3c5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:54.848277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc216636-a5a5-42c9-8f95-2b30c4f3c5cc admin admin] PATH_INFO: `/v3/projects/5c6ae91500de48abbde2a4ae868de095/users/fcd673082c9249a9af03fec8e2ae4508/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:54.877107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216636-a5a5-42c9-8f95-2b30c4f3c5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6ae91500de48abbde2a4ae868de095, user_id=fcd673082c9249a9af03fec8e2ae4508, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:54.882357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc216636-a5a5-42c9-8f95-2b30c4f3c5cc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:54.903555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d7149a3ef44255a69bfb65bca651cd, user_id=42200db3f7e74c36bcbecbdd741052ec, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:54.915619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:55.200605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 862/1744] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:54 2022] PUT /identity/v3/projects/5c6ae91500de48abbde2a4ae868de095/users/fcd673082c9249a9af03fec8e2ae4508/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:22:55.200605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cc588-14f6-4e61-8494-706a4044e8b5 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.200605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cc588-14f6-4e61-8494-706a4044e8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.200605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cc588-14f6-4e61-8494-706a4044e8b5 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.202453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 863/1745] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:55.204206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 883/1746] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:54 2022] PUT /identity/v3/projects/f1d7149a3ef44255a69bfb65bca651cd/users/42200db3f7e74c36bcbecbdd741052ec/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:55.205613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cc588-14f6-4e61-8494-706a4044e8b5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:55.208671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:55.368150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667178c7-9b87-4e0a-9298-fe6e43db3287 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:55.426034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cc588-14f6-4e61-8494-706a4044e8b5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:55.433528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beccb72-1986-4115-a319-545734dbcd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:55.434583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beccb72-1986-4115-a319-545734dbcd71 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.435272 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beccb72-1986-4115-a319-545734dbcd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.436690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beccb72-1986-4115-a319-545734dbcd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.439278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beccb72-1986-4115-a319-545734dbcd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:55.442465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beccb72-1986-4115-a319-545734dbcd71 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:55.524192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:55.525174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.525779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.527378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.530372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:55.536160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:55.548876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 864/1747] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:22:55.551181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6ff50-1229-46dd-be06-94b875116287 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.552262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6ff50-1229-46dd-be06-94b875116287 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.553343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6ff50-1229-46dd-be06-94b875116287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.585466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 884/1748] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:22:55 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:22:55.589553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:55.619014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-dcb6ff50-1229-46dd-be06-94b875116287 None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:55.627676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-dcb6ff50-1229-46dd-be06-94b875116287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:55.636552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635395dd-c63d-4d96-905b-63ab0e7e1cb5 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:55.647891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:55.651001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.651320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.653044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] PATH_INFO: `/v3/projects/5c6ae91500de48abbde2a4ae868de095/users/fcd673082c9249a9af03fec8e2ae4508/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.658140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c6ae91500de48abbde2a4ae868de095, user_id=fcd673082c9249a9af03fec8e2ae4508, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:55.665169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:55.714500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 885/1749] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:55 2022] PUT /identity/v3/projects/5c6ae91500de48abbde2a4ae868de095/users/fcd673082c9249a9af03fec8e2ae4508/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:55.747081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 865/1750] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79597088-4079-45da-9b38-860b29062824 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79597088-4079-45da-9b38-860b29062824 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79597088-4079-45da-9b38-860b29062824 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 866/1751] 10.222.0.13 () {54 vars in 912 bytes} [Wed Aug 10 11:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.973643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.975466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.975466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] MFA Rules not processed for user `fcd673082c9249a9af03fec8e2ae4508`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:55.975466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:55.975730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2662b42-43b6-43d4-a852-3fd9816055ca admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:55.979022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e7da3-488a-47ba-b071-6b6d26a791b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:55.980239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e7da3-488a-47ba-b071-6b6d26a791b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:55.980815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e7da3-488a-47ba-b071-6b6d26a791b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:55.981358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e7da3-488a-47ba-b071-6b6d26a791b3 admin admin] PATH_INFO: `/v3/projects/f1d7149a3ef44255a69bfb65bca651cd/users/42200db3f7e74c36bcbecbdd741052ec/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:55.984614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e7da3-488a-47ba-b071-6b6d26a791b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1d7149a3ef44255a69bfb65bca651cd, user_id=42200db3f7e74c36bcbecbdd741052ec, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:55.989379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e7da3-488a-47ba-b071-6b6d26a791b3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.008483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 886/1752] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:22:55 2022] PUT /identity/v3/projects/f1d7149a3ef44255a69bfb65bca651cd/users/42200db3f7e74c36bcbecbdd741052ec/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:22:56.010106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba529eb2-341a-4b58-bdea-3b988648e67d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.010619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba529eb2-341a-4b58-bdea-3b988648e67d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.011130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba529eb2-341a-4b58-bdea-3b988648e67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.057769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ba529eb2-341a-4b58-bdea-3b988648e67d None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:56.059189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 867/1753] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:55 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:56.060148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.060278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ba529eb2-341a-4b58-bdea-3b988648e67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.063454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.202363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 887/1754] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:56.202363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.202363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.202363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.220136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b7678-6b92-4a35-8be5-9633f6a7ac58 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.259558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] MFA Rules not processed for user `42200db3f7e74c36bcbecbdd741052ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:56.261067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.397986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.401011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.401840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.402308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.429457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.431520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.471847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 888/1755] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:56.473057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.478007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.478390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 868/1756] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:56.481000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.497400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:56.503366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8678d-86e6-426e-b8bf-6b7fc4f82210 service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.508346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.509194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.509804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.510193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.520298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.527324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.539307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 869/1757] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:56.540312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:56.542305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:56.582807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:56.620732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c4ce8-896f-4e14-9916-861e78b988b9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.651204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.651204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.651204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.658385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.718124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d55dad-b890-4d2b-b1aa-7b97443a5551 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:56.874100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:56.874100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:56.874100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:56.874100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:56.874686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.874686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.874686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 870/1758] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:56.911065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:56.920980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:56.954654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 889/1759] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:56 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:57.087434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:57.105032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1396c8c2-fbb7-4283-9b47-9947cbd1c869 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:57.115048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88fc69-fe9b-4250-b42f-ec15ea5f05cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:57.126040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88fc69-fe9b-4250-b42f-ec15ea5f05cd service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.126396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88fc69-fe9b-4250-b42f-ec15ea5f05cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.126679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88fc69-fe9b-4250-b42f-ec15ea5f05cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.170889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:57.184678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11187e55-4cce-4d42-a238-a5106561230c service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:57.213453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce2066-3b4b-43c7-a7f4-5160fef6844d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:57.215032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce2066-3b4b-43c7-a7f4-5160fef6844d service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.215473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce2066-3b4b-43c7-a7f4-5160fef6844d service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.215901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce2066-3b4b-43c7-a7f4-5160fef6844d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.298085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88fc69-fe9b-4250-b42f-ec15ea5f05cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:57.300502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad88fc69-fe9b-4250-b42f-ec15ea5f05cd service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:57.322834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 871/1760] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:57.377211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce2066-3b4b-43c7-a7f4-5160fef6844d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:57.383681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce2066-3b4b-43c7-a7f4-5160fef6844d service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:57.392258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 890/1761] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:57.873552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d55b5-fac5-47da-975b-8dddbf597022 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.873900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d55b5-fac5-47da-975b-8dddbf597022 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.874100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d55b5-fac5-47da-975b-8dddbf597022 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.875666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 872/1762] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:22:57.894772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f478c1b-4a72-4907-b1e3-4741e1377cf7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.895328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f478c1b-4a72-4907-b1e3-4741e1377cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.895328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f478c1b-4a72-4907-b1e3-4741e1377cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:57.927696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4f478c1b-4a72-4907-b1e3-4741e1377cf7 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:57.932107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4f478c1b-4a72-4907-b1e3-4741e1377cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:57.984138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:57.984425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:57.984720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:58.052280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] MFA Rules not processed for user `0782e27e2318438fbf155dfa9cb834b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:58.053810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:58.145041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 891/1763] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:22:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:58.146113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:58.146462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:58.146664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:58.228949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] MFA Rules not processed for user `87adb637dd814cf8be9ff6c2a2bc26ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:22:58.230955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:58.267192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 873/1764] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:22:58.473154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 892/1765] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:22:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:22:58.746567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:58.746567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:58.768932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09912c75-0655-4d22-acfc-27f985d69302 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:58.789501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:58.790449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:58.790787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:58.791061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:58.854318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:58.862163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:22:58.960530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 874/1766] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:22:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:22:59.181071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:59.515678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:22:59.541001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:22:59.542611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:22:59.748667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007cfe7-fc5e-42a8-904c-e5f794a8da92 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:22:59.772486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:22:59.774772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:22:59.775105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:22:59.775379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] PATH_INFO: `/v3/users/3e48377440f445e2b55682bd1161811c` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:22:59.777722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e48377440f445e2b55682bd1161811c)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:22:59.780730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:00.159673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e48377440f445e2b55682bd1161811c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:00.161515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e48377440f445e2b55682bd1161811c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:00.162986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e48377440f445e2b55682bd1161811c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:00.184574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e48377440f445e2b55682bd1161811c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:00.268326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e48377440f445e2b55682bd1161811c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:00.306821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 893/1767] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:22:59 2022] DELETE /identity/v3/users/3e48377440f445e2b55682bd1161811c => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:00.722940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:01.356778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:01.642541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:02.549953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:02.564396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339fa76-e4f3-4381-9b0d-cf9542e46c9c service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:02.659320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:02.664064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:02.664064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:02.664064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] PATH_INFO: `/v3/projects/855fec46478349ea978f0174c643ff06` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:02.664064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=855fec46478349ea978f0174c643ff06)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:02.664872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:02.831950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '855fec46478349ea978f0174c643ff06', 'request_id': 'req-0a0b7821-2331-485f-9f7a-12bd53d970f7'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:02.849652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '855fec46478349ea978f0174c643ff06', 'request_id': 'req-0a0b7821-2331-485f-9f7a-12bd53d970f7'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:02.884347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '855fec46478349ea978f0174c643ff06', 'request_id': 'req-0a0b7821-2331-485f-9f7a-12bd53d970f7'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.043330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] The token cache is being invalidate because project 855fec46478349ea978f0174c643ff06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:03.043721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.044426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0a0b7821-2331-485f-9f7a-12bd53d970f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.049320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 875/1768] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:02 2022] DELETE /identity/v3/projects/855fec46478349ea978f0174c643ff06 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:03.050862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:03.052544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:03.081802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba5371-ef9f-4983-8c3a-9cc9fc60250a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:03.192063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:03.193023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:03.193511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:03.193782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] PATH_INFO: `/v3/users/4fe6d4cfa16146598e9cba54abed2733` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:03.194998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe6d4cfa16146598e9cba54abed2733)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:03.197735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:03.369941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639428-3ec5-4a53-8b04-58f066a8273d None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:03.374947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639428-3ec5-4a53-8b04-58f066a8273d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:03.374947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c639428-3ec5-4a53-8b04-58f066a8273d None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:03.374947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 876/1769] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:23:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:23:03.389564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:03.389564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:03.389823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:03.432045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:03.434043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:03.459239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:03.465374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe6d4cfa16146598e9cba54abed2733'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.467328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe6d4cfa16146598e9cba54abed2733'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.471544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe6d4cfa16146598e9cba54abed2733'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.483926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe6d4cfa16146598e9cba54abed2733'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.534089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 877/1770] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:23:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:03.552558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:03.554111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:03.571682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46be40-b0b7-4afd-811b-66f5edef7889 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:03.580499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:03.581140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:03.581425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:03.581692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:03.600083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:03.600083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:03.611880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 878/1771] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:03.633793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe6d4cfa16146598e9cba54abed2733'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:03.700148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 894/1772] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:03 2022] DELETE /identity/v3/users/4fe6d4cfa16146598e9cba54abed2733 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:03.717437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:04.729735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:04.734236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:04.760747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6c675-f608-45d3-82f5-d94c822eb24b service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:04.819808 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:04.820524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:04.820804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:04.821084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] PATH_INFO: `/v3/users/a5c003e4470a46ce90f3f01b03ed5766` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:04.822400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5c003e4470a46ce90f3f01b03ed5766)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:04.824609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:04.913002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:04.932652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:04.932652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fb431-2e86-4115-a6d1-3aa8f258a8f3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:04.932652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:04.933993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:04.933993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:04.933993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] PATH_INFO: `/v3/projects/14b9b464fc4e410ebab1fdacc0da1286` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:04.934576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b9b464fc4e410ebab1fdacc0da1286)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:04.937181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:05.112754 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5c003e4470a46ce90f3f01b03ed5766'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.113360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5c003e4470a46ce90f3f01b03ed5766'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.113964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5c003e4470a46ce90f3f01b03ed5766'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.121204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b9b464fc4e410ebab1fdacc0da1286', 'request_id': 'req-a5626c9c-b5db-4a40-902a-f01ea424e8b2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.121967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5c003e4470a46ce90f3f01b03ed5766'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.127981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b9b464fc4e410ebab1fdacc0da1286', 'request_id': 'req-a5626c9c-b5db-4a40-902a-f01ea424e8b2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.134909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b9b464fc4e410ebab1fdacc0da1286', 'request_id': 'req-a5626c9c-b5db-4a40-902a-f01ea424e8b2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.247582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-be0a4748-de61-48e4-aa96-dbcb57485987 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5c003e4470a46ce90f3f01b03ed5766'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.266853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 879/1773] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:04 2022] DELETE /identity/v3/users/a5c003e4470a46ce90f3f01b03ed5766 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:05.271178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.271464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.271736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.291483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:05.292727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:05.337609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] The token cache is being invalidate because project 14b9b464fc4e410ebab1fdacc0da1286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:05.338073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.338902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a5626c9c-b5db-4a40-902a-f01ea424e8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:05.340677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 895/1774] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:04 2022] DELETE /identity/v3/projects/14b9b464fc4e410ebab1fdacc0da1286 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:05.350466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 880/1775] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:05.356948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8e2cb-d0f0-4475-8fd4-13f16a12b1f2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.373757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:05.373757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8e2cb-d0f0-4475-8fd4-13f16a12b1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.373757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8e2cb-d0f0-4475-8fd4-13f16a12b1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.373972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:05.375301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:05.437286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-3da8e2cb-d0f0-4475-8fd4-13f16a12b1f2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:05.438847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3da8e2cb-d0f0-4475-8fd4-13f16a12b1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:05.461954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 896/1776] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:05.464125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.464571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.464985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.509820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:05.512095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:05.517018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6630a8-9549-4f1b-93ce-a207336a6d61 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:05.520649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:05.521252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.521686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.521966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.527119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 897/1777] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:05.530414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:05.541525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e564-0649-45bf-a0d3-fff193643765 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:05.545837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:05.547666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:05.547806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:05.548399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.548679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.548949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.549808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:05.550264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:05.552462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:05.562107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 881/1778] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:05.570669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 898/1779] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:05.580593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:05.599390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47cdd64-5d9f-438e-9ddb-c41be023b9ae service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:05.605344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:05.607304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.608090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.608090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.608583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:05.610585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:05.737390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 882/1780] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:05 2022] POST /identity/v3/projects => generated 365 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:05.741934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:05.754372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ad1a-92f3-46f8-a176-831301da2215 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:05.838209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:05.838698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:05.839069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:05.839324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:05.839949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:05.842512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:05.978983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 899/1781] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:05 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:05.983786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:05.994014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d5656-1a8a-4f5f-9a3d-63cfa495aad6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:05.998907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.010388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.010388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.010388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.010388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.010388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.012483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 883/1782] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:06.016734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.026550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ed53-1383-4527-ba24-f6200b5d8fa2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.031863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.032484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.032755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.033022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.033650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.035230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.046333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 900/1783] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:06.050297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.060095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc9a37-9e30-4274-9a32-2e31688f412d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.064978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aea897-e376-4b8c-aa81-de083c1917e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.065623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aea897-e376-4b8c-aa81-de083c1917e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.065900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aea897-e376-4b8c-aa81-de083c1917e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.066169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aea897-e376-4b8c-aa81-de083c1917e8 admin admin] PATH_INFO: `/v3/projects/9c70a7a3fcf6408c9539c913c4b4140e/users/d28293d25e2b40a4a992c3beffe1faad/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.080854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aea897-e376-4b8c-aa81-de083c1917e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c70a7a3fcf6408c9539c913c4b4140e, user_id=d28293d25e2b40a4a992c3beffe1faad, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.083472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aea897-e376-4b8c-aa81-de083c1917e8 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.095499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.097278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:06.110499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76ef685-1fd6-4524-a484-0dc308b191f9 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.115342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.115976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.116291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.120533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] PATH_INFO: `/v3/projects/a063e909b4544557993de6d039b22eee` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.120533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a063e909b4544557993de6d039b22eee)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.120533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.194187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 884/1784] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:06 2022] PUT /identity/v3/projects/9c70a7a3fcf6408c9539c913c4b4140e/users/d28293d25e2b40a4a992c3beffe1faad/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:06.206667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.206952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.207227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.234581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] MFA Rules not processed for user `d28293d25e2b40a4a992c3beffe1faad`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:06.235711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:06.265049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a063e909b4544557993de6d039b22eee', 'request_id': 'req-89648cd6-2570-47c9-bec7-6b488188eb6e'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.270463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a063e909b4544557993de6d039b22eee', 'request_id': 'req-89648cd6-2570-47c9-bec7-6b488188eb6e'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.279867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a063e909b4544557993de6d039b22eee', 'request_id': 'req-89648cd6-2570-47c9-bec7-6b488188eb6e'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.306417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 885/1785] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:06.309021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.373117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654d48db-8a2a-4a66-bd69-4270be44f40f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.374754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] The token cache is being invalidate because project a063e909b4544557993de6d039b22eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:06.375048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.375706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.377465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 901/1786] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:06 2022] DELETE /identity/v3/projects/a063e909b4544557993de6d039b22eee => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:06.378020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.381065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:06.438364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.438992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.439274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.439612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.440372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.442326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.460735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89648cd6-2570-47c9-bec7-6b488188eb6e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.466009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.467406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.467773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.468222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] PATH_INFO: `/v3/users/f44bf26e27c546d085a0ed7cf102f105` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.469634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44bf26e27c546d085a0ed7cf102f105)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.471598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.564239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 886/1787] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:06 2022] POST /identity/v3/projects => generated 367 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:06.565399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.567170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:06.635514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24ccc7-63f1-4caf-a847-3a89957491b2 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.642112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:06.642751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:06.643072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:06.643367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] PATH_INFO: `/v3/users/42200db3f7e74c36bcbecbdd741052ec` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:06.644530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42200db3f7e74c36bcbecbdd741052ec)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:06.646552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:06.702962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44bf26e27c546d085a0ed7cf102f105'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.703605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44bf26e27c546d085a0ed7cf102f105'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.704244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44bf26e27c546d085a0ed7cf102f105'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.711160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44bf26e27c546d085a0ed7cf102f105'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.837267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44bf26e27c546d085a0ed7cf102f105'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.864112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 902/1788] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:06 2022] DELETE /identity/v3/users/f44bf26e27c546d085a0ed7cf102f105 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:06.864847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:06.866479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:06.923621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42200db3f7e74c36bcbecbdd741052ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.924574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42200db3f7e74c36bcbecbdd741052ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.926485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42200db3f7e74c36bcbecbdd741052ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.948733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42200db3f7e74c36bcbecbdd741052ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:06.993910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87432f1-3769-4b29-8b7d-dca9d5042a40 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:06.999861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e4483-40fe-4c09-ba28-06f826f5bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.000675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4483-40fe-4c09-ba28-06f826f5bdfa admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.000968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4483-40fe-4c09-ba28-06f826f5bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.001290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e4483-40fe-4c09-ba28-06f826f5bdfa admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.001922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4483-40fe-4c09-ba28-06f826f5bdfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.005215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e4483-40fe-4c09-ba28-06f826f5bdfa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.072117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42200db3f7e74c36bcbecbdd741052ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:07.119904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 903/1789] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:06 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:07.123949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.127590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.127882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.134150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 887/1790] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:06 2022] DELETE /identity/v3/users/42200db3f7e74c36bcbecbdd741052ec => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:07.134150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.141118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.182876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:07.184839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.308107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3205e-0fb6-43a4-8149-b085182236b0 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.313274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 904/1791] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:07.321796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.322526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.322795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.323070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.324119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.327457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.343244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 888/1792] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:07.350791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.351938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.364097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.386562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152483af-42f8-4c36-a9e7-bec997ad9dba admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.393907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.394484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.394761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.395372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] PATH_INFO: `/v3/projects/c88f18a655b54ccbb7fbc92cdb4cb567/users/c153c898e5804b3392234d1e5b026366/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.417678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88f18a655b54ccbb7fbc92cdb4cb567, user_id=c153c898e5804b3392234d1e5b026366, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.421484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.490424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec0f7b-590a-48e2-93dc-c497d381de4d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.495363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.496098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.496383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.496658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.506444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.507861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.517652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 905/1793] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:07.531987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 889/1794] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:07 2022] PUT /identity/v3/projects/c88f18a655b54ccbb7fbc92cdb4cb567/users/c153c898e5804b3392234d1e5b026366/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:07.536396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.548283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888711f5-ebb9-4f2b-ba55-77ee1e15fe95 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.618143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a6e677-1522-4f27-bbd3-8d76cf535ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.619114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6e677-1522-4f27-bbd3-8d76cf535ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.619668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6e677-1522-4f27-bbd3-8d76cf535ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.619944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a6e677-1522-4f27-bbd3-8d76cf535ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.621378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6e677-1522-4f27-bbd3-8d76cf535ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.622533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a6e677-1522-4f27-bbd3-8d76cf535ba3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.636838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 906/1795] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:07 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:07.638260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.650435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc257-d9ed-460d-ad6e-5f01af4d4b7a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:07.702105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:07.703650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.707451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:07.708096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.708918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] PATH_INFO: `/v3/projects/c88f18a655b54ccbb7fbc92cdb4cb567/users/c153c898e5804b3392234d1e5b026366/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.723567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c88f18a655b54ccbb7fbc92cdb4cb567, user_id=c153c898e5804b3392234d1e5b026366, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:07.735163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:07.773707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 890/1796] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:07 2022] PUT /identity/v3/projects/c88f18a655b54ccbb7fbc92cdb4cb567/users/c153c898e5804b3392234d1e5b026366/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:07.776306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:07.776852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:07.777296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:07.818020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] MFA Rules not processed for user `c153c898e5804b3392234d1e5b026366`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:07.820538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:07.976794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 907/1797] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:07.990281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:07.993012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:08.085407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5b991-95cb-4553-bbd3-aab96561762c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.090471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.092185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.097007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.097007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.104946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.108118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.138016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 891/1798] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:08.415151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.436260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011b5b3-03c6-4d84-b278-8820261077f7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.440437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.441754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.441975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.442259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.444757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:08.563631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:08.565393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:08.581836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.583073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba529a24-76f2-49b9-bbe0-611a213830ab service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:08.608789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 908/1799] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:08.713626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74697461-c7dc-44b6-80f1-e372f3ad379d service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:08.728990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:08.730011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:08.730291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:08.730615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] PATH_INFO: `/v3/projects/f1d7149a3ef44255a69bfb65bca651cd` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:08.734870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1d7149a3ef44255a69bfb65bca651cd)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:08.737328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.002015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1d7149a3ef44255a69bfb65bca651cd', 'request_id': 'req-791e8d3d-2a69-4bdc-90ea-59f34a114d62'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.037178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1d7149a3ef44255a69bfb65bca651cd', 'request_id': 'req-791e8d3d-2a69-4bdc-90ea-59f34a114d62'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.053304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1d7149a3ef44255a69bfb65bca651cd', 'request_id': 'req-791e8d3d-2a69-4bdc-90ea-59f34a114d62'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.165360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] The token cache is being invalidate because project f1d7149a3ef44255a69bfb65bca651cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:09.167690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.168468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:09.170446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.175043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 892/1800] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:08 2022] DELETE /identity/v3/projects/f1d7149a3ef44255a69bfb65bca651cd => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:09.180692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2060b-0516-4e3d-a00a-18cc5ab0abce None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.181112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2060b-0516-4e3d-a00a-18cc5ab0abce None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.181547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce2060b-0516-4e3d-a00a-18cc5ab0abce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.222723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fce2060b-0516-4e3d-a00a-18cc5ab0abce None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:09.224645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fce2060b-0516-4e3d-a00a-18cc5ab0abce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:09.301004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.301004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:09.338105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 909/1801] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:09.347175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.351026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.351558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.395542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791e8d3d-2a69-4bdc-90ea-59f34a114d62 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.409589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.412128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.413391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.414875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] PATH_INFO: `/v3/projects/6e06f66aefac4b0f8a5a6df17a217888` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.416100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:09.416884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e06f66aefac4b0f8a5a6df17a217888)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.417321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:09.419713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.441163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 910/1802] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:09.446222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.460821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d14c31-6398-42c2-8952-6b7fed8787ad None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.468369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.469230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.469588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.469930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.470761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:09.471135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.473867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.493864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 911/1803] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:23:09.495116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.507511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35cd3a-1dfb-4b26-9632-31c20b467127 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.512896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.514717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.515075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.515562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.516366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.518538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.634336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e06f66aefac4b0f8a5a6df17a217888', 'request_id': 'req-81252ecd-b9d8-48e8-99e4-ba9440252cdc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.644810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e06f66aefac4b0f8a5a6df17a217888', 'request_id': 'req-81252ecd-b9d8-48e8-99e4-ba9440252cdc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.667018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e06f66aefac4b0f8a5a6df17a217888', 'request_id': 'req-81252ecd-b9d8-48e8-99e4-ba9440252cdc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.667670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 912/1804] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:09 2022] POST /identity/v3/projects => generated 387 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:09.672966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.776181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] The token cache is being invalidate because project 6e06f66aefac4b0f8a5a6df17a217888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:09.776681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.777427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:09.783029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 893/1805] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:09 2022] DELETE /identity/v3/projects/6e06f66aefac4b0f8a5a6df17a217888 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:09.784250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:09.785877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:09.836758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ed5e64-c3c8-45a8-b05e-73bd52bcc422 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.841899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.842706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.842992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.843268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.843900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.846564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:09.876545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81252ecd-b9d8-48e8-99e4-ba9440252cdc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:09.881610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:09.882259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:09.882542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:09.882938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] PATH_INFO: `/v3/users/fcd673082c9249a9af03fec8e2ae4508` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:09.884296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcd673082c9249a9af03fec8e2ae4508)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:09.886225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:10.010095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 913/1806] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:09 2022] POST /identity/v3/users => generated 294 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:10.010892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:10.013977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:10.059429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b691f-6fb1-4d11-b0dc-e2b87c44f93a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:10.066469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:10.067344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:10.068150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:10.068150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:10.069184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:10.071141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:10.083036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 914/1807] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:10.088169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:10.098401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31896588-0635-4feb-b856-e7eed31a836a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:10.104960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:10.104960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:10.104960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:10.105221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:10.108194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:10.108194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:10.124977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 915/1808] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:10.129686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:10.140579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5322a20-a196-4437-9551-8779c24d2133 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:10.146026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1244b9e-036f-45cb-b9ab-61d7558abc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:10.146786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1244b9e-036f-45cb-b9ab-61d7558abc4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:10.147064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1244b9e-036f-45cb-b9ab-61d7558abc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:10.147384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1244b9e-036f-45cb-b9ab-61d7558abc4a admin admin] PATH_INFO: `/v3/projects/6a57c8a84103478ba18cab6cb1c064d0/users/12872716069e4fa88a45482ff3a67a8b/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:10.163222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1244b9e-036f-45cb-b9ab-61d7558abc4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a57c8a84103478ba18cab6cb1c064d0, user_id=12872716069e4fa88a45482ff3a67a8b, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:10.169268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1244b9e-036f-45cb-b9ab-61d7558abc4a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:10.184804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcd673082c9249a9af03fec8e2ae4508'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:10.186927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcd673082c9249a9af03fec8e2ae4508'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:10.187557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcd673082c9249a9af03fec8e2ae4508'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:10.196066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcd673082c9249a9af03fec8e2ae4508'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:10.298694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcd673082c9249a9af03fec8e2ae4508'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:10.311942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 916/1809] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:10 2022] PUT /identity/v3/projects/6a57c8a84103478ba18cab6cb1c064d0/users/12872716069e4fa88a45482ff3a67a8b/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:10.321708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:10.322374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:10.324275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:10.339539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 894/1810] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:09 2022] DELETE /identity/v3/users/fcd673082c9249a9af03fec8e2ae4508 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:10.370673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] MFA Rules not processed for user `12872716069e4fa88a45482ff3a67a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:10.372377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:10.469964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:10.472381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:10.477316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 917/1811] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:10.580482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ded82-0237-43d7-8db3-0d0e7abe87c1 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:10.593598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:10.594906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:10.595480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:10.595924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:10.597159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:10.599618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:10.626673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:10.764787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:10.766143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 895/1812] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:10 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:10.803682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f21b724-5aa7-400d-913d-0c9be59510dd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:10.925795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:10.926710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:10.926981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:10.927245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:10.929017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:10.930087 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.101429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 918/1813] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:10 2022] POST /identity/v3/users => generated 300 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:11.102597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.137660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a3a95-7472-48b8-b4ca-78d1892931db admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.157812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37642bf4-06c2-4183-b31f-50066edef816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.158470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37642bf4-06c2-4183-b31f-50066edef816 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.159236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37642bf4-06c2-4183-b31f-50066edef816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.159236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37642bf4-06c2-4183-b31f-50066edef816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.160360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37642bf4-06c2-4183-b31f-50066edef816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.161974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37642bf4-06c2-4183-b31f-50066edef816 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.178956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 896/1814] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:11.182889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.195193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2e53a-1362-4e8c-9740-2a2686ff8da7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.201699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.202675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.203145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.203685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] PATH_INFO: `/v3/projects/9e23d7e22aab4a08a7dcb51862aff52b/users/690fc85a490141de87f6755b928f729c/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.223441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e23d7e22aab4a08a7dcb51862aff52b, user_id=690fc85a490141de87f6755b928f729c, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.227824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.250198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92bbf-f3fb-4efb-a060-a9cf7392119f None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.250770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92bbf-f3fb-4efb-a060-a9cf7392119f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.250770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b92bbf-f3fb-4efb-a060-a9cf7392119f None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.252063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 897/1815] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:23:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:23:11.258309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.258572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.258848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.280629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:11.282851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.325489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 919/1816] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:11 2022] PUT /identity/v3/projects/9e23d7e22aab4a08a7dcb51862aff52b/users/690fc85a490141de87f6755b928f729c/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:11.330414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.349367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 898/1817] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:23:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:11.359378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:11.361307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.363263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.384774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a3064d-a0ff-4c90-8522-c361076af173 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.421826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ac6fc4-6a1e-4b60-b624-adc288ee0fbc admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.427790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.428432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.428796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.429119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.429792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.431429 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.443494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 920/1818] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:11.443494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.443494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.453319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.453967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.454247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.454524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] PATH_INFO: `/v3/projects/5c6ae91500de48abbde2a4ae868de095` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.455650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c6ae91500de48abbde2a4ae868de095)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.457698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.524099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafedadf-f552-47d5-a5a8-35895a627b15 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.530164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.530787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.531068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.531348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.540798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.542374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.553394 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 921/1819] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:11.554831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.566919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6d8aa7-cdd4-486d-8e5f-42d2dbe1ebb0 service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.585393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c6ae91500de48abbde2a4ae868de095', 'request_id': 'req-b5c68a36-46a8-482f-9397-ef3d7d4421c0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:11.592625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c6ae91500de48abbde2a4ae868de095', 'request_id': 'req-b5c68a36-46a8-482f-9397-ef3d7d4421c0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:11.598504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c6ae91500de48abbde2a4ae868de095', 'request_id': 'req-b5c68a36-46a8-482f-9397-ef3d7d4421c0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:11.627878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c3fbd-54e9-4951-bb6a-434f5b0d7f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.628567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c3fbd-54e9-4951-bb6a-434f5b0d7f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.628846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c3fbd-54e9-4951-bb6a-434f5b0d7f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.629118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c3fbd-54e9-4951-bb6a-434f5b0d7f94 admin admin] PATH_INFO: `/v3/projects/9e23d7e22aab4a08a7dcb51862aff52b/users/690fc85a490141de87f6755b928f729c/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.631034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c3fbd-54e9-4951-bb6a-434f5b0d7f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e23d7e22aab4a08a7dcb51862aff52b, user_id=690fc85a490141de87f6755b928f729c, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.633693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0c3fbd-54e9-4951-bb6a-434f5b0d7f94 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.643593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 922/1820] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:11 2022] PUT /identity/v3/projects/9e23d7e22aab4a08a7dcb51862aff52b/users/690fc85a490141de87f6755b928f729c/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:11.653540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.653916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.654372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.680938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] The token cache is being invalidate because project 5c6ae91500de48abbde2a4ae868de095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:11.681330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:11.681987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b5c68a36-46a8-482f-9397-ef3d7d4421c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:11.682111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] MFA Rules not processed for user `690fc85a490141de87f6755b928f729c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:11.683369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.683830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 899/1821] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:11 2022] DELETE /identity/v3/projects/5c6ae91500de48abbde2a4ae868de095 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:11.689363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78720a72-22c8-4d3e-aee8-215ecd6e6bd0 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.689717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78720a72-22c8-4d3e-aee8-215ecd6e6bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.690062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78720a72-22c8-4d3e-aee8-215ecd6e6bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.708262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:11.713764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-78720a72-22c8-4d3e-aee8-215ecd6e6bd0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:11.715001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-78720a72-22c8-4d3e-aee8-215ecd6e6bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.779407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 923/1822] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:11.790245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.792038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.802370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 900/1823] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:11.809840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.810695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.811885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.833595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:11.834812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:11.848181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 901/1824] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:11.852306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.866234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34bc0b-4908-442b-ad6e-074bd34cf84f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.868507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e883bf6-ceca-403f-b6d4-22df42e53380 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.870552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.871291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.871596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.871865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.874713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.875328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.875606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.875879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.876563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:11.876866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.878539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.884379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.886386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:11.895203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 902/1825] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:11.900812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:11.908187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 924/1826] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:11.913942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a2ab4-0f12-454b-8d09-5bc796fb66ea admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:11.926615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:11.927559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:11.927840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:11.928400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:11.928877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:11.932680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:12.096900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 903/1827] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:11 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:12.104561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:12.116584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5344956-d6a6-4886-963a-fd40d25e5510 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:12.207437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:12.208474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:12.208826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:12.209186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:12.210024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:12.214122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:12.365321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 925/1828] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:12 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:12.366692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:12.380117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68905148-0b65-49dc-9fa4-d50b49739aea admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:12.389478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:12.390190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:12.390693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:12.391420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:12.405275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:12.408100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:12.426966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 904/1829] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:12.430711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:12.444424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b46130-82ad-41d8-abc2-ea8cd69c9e0d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:12.454050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8c4c9-cdb0-4aa6-b027-1de4c71a2d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:12.454759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c4c9-cdb0-4aa6-b027-1de4c71a2d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:12.455106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c4c9-cdb0-4aa6-b027-1de4c71a2d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:12.455440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8c4c9-cdb0-4aa6-b027-1de4c71a2d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:12.456209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8c4c9-cdb0-4aa6-b027-1de4c71a2d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:12.458168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c8c4c9-cdb0-4aa6-b027-1de4c71a2d68 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:12.473677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 926/1830] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:12.478431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:12.502247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2b60c-00b7-4259-a10a-34265165275f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:12.516523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:12.517202 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:12.517486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:12.517764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] PATH_INFO: `/v3/projects/2941567e527a4cf9974002c690191d1a/users/c690fa446a8046d498eb41b916cb431e/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:12.546307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2941567e527a4cf9974002c690191d1a, user_id=c690fa446a8046d498eb41b916cb431e, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:12.549732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:12.663981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:12.672505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 905/1831] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:12 2022] PUT /identity/v3/projects/2941567e527a4cf9974002c690191d1a/users/c690fa446a8046d498eb41b916cb431e/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:12.686898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:12.687238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:12.687574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:12.719582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] MFA Rules not processed for user `c690fa446a8046d498eb41b916cb431e`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:12.721506 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:12.858400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 927/1832] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:12.861466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:12.976046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c116d584-f425-4ab3-8a2d-fffccc11f80c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:12.985281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:12.986126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:12.986498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:12.986862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:12.987583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:12.990645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.118446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 906/1833] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:12 2022] POST /identity/v3/projects => generated 379 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:13.122341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.137272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447a9a4-8b19-4ae2-a9c3-f45a605fbfda None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.223427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.224082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.224365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.224637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.225298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.227207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.265213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:13.297984 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.299901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:13.331756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65209c00-a9cd-4319-8b18-1b097485fd04 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.340415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 928/1834] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:13 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:13.342142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.346996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.348604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.349075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.349723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] PATH_INFO: `/v3/users/12872716069e4fa88a45482ff3a67a8b` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.352199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12872716069e4fa88a45482ff3a67a8b)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.355159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.367138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2f1e6-cb99-4934-8fdb-8e38eadc9fe7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.379374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.380132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.380407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.380900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.381746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.383712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.396247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 929/1835] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:13.400395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.410434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88917e1b-fb99-4e97-bdd2-613465ce0ab3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.415095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.415703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.415983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.416297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.416935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.418523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.432050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 930/1836] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:13.433797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.444297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066932e-7108-46af-a120-56749d5a0e90 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.449380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c1b96b-4812-4aa9-b382-066e94bb0a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.450037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1b96b-4812-4aa9-b382-066e94bb0a8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.450315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1b96b-4812-4aa9-b382-066e94bb0a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.450610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c1b96b-4812-4aa9-b382-066e94bb0a8b admin admin] PATH_INFO: `/v3/projects/64d9ea8b5e4f47eeaa007f5268ceaf7f/users/738dcba7096b44d492653def32c2ea6d/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.469131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1b96b-4812-4aa9-b382-066e94bb0a8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d9ea8b5e4f47eeaa007f5268ceaf7f, user_id=738dcba7096b44d492653def32c2ea6d, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.470614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c1b96b-4812-4aa9-b382-066e94bb0a8b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.567342 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 931/1837] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:13 2022] PUT /identity/v3/projects/64d9ea8b5e4f47eeaa007f5268ceaf7f/users/738dcba7096b44d492653def32c2ea6d/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:13.572493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.572774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.573080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.574950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12872716069e4fa88a45482ff3a67a8b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:13.578295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12872716069e4fa88a45482ff3a67a8b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:13.578295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12872716069e4fa88a45482ff3a67a8b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:13.586533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12872716069e4fa88a45482ff3a67a8b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:13.615674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] MFA Rules not processed for user `738dcba7096b44d492653def32c2ea6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:13.616850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:13.689996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 932/1838] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:13.694401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.695754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:13.705358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c21b7fd8-c277-4eb8-9787-db6f645a8610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12872716069e4fa88a45482ff3a67a8b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:13.742853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 907/1839] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:13 2022] DELETE /identity/v3/users/12872716069e4fa88a45482ff3a67a8b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:13.747425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.748447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.749585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.786729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:13.788062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:13.851791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f76e67-00fd-41a7-8d50-a7c11fd3ee95 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.884029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 908/1840] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:13.885113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:13.886576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:13.916980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.917658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.918232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.918519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.919154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.922047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.946851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a07db-c602-4378-8ee6-1bdfe3863c0b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:13.951073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:13.953895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:13.953895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:13.953895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:13.962053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:13.963581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:13.973713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 909/1841] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:14.083135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 933/1842] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:13 2022] POST /identity/v3/projects => generated 377 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:14.084308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:14.095011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ae216-e011-48da-8992-e2ec03845af1 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:14.149122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:14.149729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.150011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.152152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.152152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:14.152880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:14.265331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 910/1843] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:14 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:14.269587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:14.280326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c90231-0a76-4e6b-a0cd-d550b64f7745 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:14.285323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:14.285930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.286207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.286479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.287143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:14.288768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:14.300536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 934/1844] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:14.304604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:14.314680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd49a88-9122-416c-8638-c30d1e807a5a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:14.319736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:14.320435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.320718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.320994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] PATH_INFO: `/v3/projects/cf24b8ca948f4d8b93020371e7e5eff2/users/0ddf1eaa49fa4708873106eddd11b178/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.357757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf24b8ca948f4d8b93020371e7e5eff2, user_id=0ddf1eaa49fa4708873106eddd11b178, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:14.361758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:14.477177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:14.478672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:14.498208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 911/1845] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:14 2022] PUT /identity/v3/projects/cf24b8ca948f4d8b93020371e7e5eff2/users/0ddf1eaa49fa4708873106eddd11b178/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:14.502065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:14.508726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b854e1b0-2ed0-4d59-b368-da143bab95ae admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:14.651161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53944df8-9d85-4d20-adab-c24e2af9cf8b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:14.653536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:14.654881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.655227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.655946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] PATH_INFO: `/v3/projects/6a57c8a84103478ba18cab6cb1c064d0` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.657569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a57c8a84103478ba18cab6cb1c064d0)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:14.659953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:14.661451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:14.663568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.664846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.665618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.667256 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:14.672336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:14.691924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 912/1846] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:14.693312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:14.726059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad31b6-43a1-429a-b5a2-8c726a6b2982 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:14.737695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656505f3-9183-4643-9b56-ec0c08f97938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:14.738391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656505f3-9183-4643-9b56-ec0c08f97938 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.738673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656505f3-9183-4643-9b56-ec0c08f97938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.738949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656505f3-9183-4643-9b56-ec0c08f97938 admin admin] PATH_INFO: `/v3/projects/cf24b8ca948f4d8b93020371e7e5eff2/users/0ddf1eaa49fa4708873106eddd11b178/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.743813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656505f3-9183-4643-9b56-ec0c08f97938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf24b8ca948f4d8b93020371e7e5eff2, user_id=0ddf1eaa49fa4708873106eddd11b178, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:14.746559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656505f3-9183-4643-9b56-ec0c08f97938 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:14.785104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 913/1847] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:14 2022] PUT /identity/v3/projects/cf24b8ca948f4d8b93020371e7e5eff2/users/0ddf1eaa49fa4708873106eddd11b178/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:14.795303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:14.811819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:14.812238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:14.881530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] MFA Rules not processed for user `0ddf1eaa49fa4708873106eddd11b178`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:14.883411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:14.890425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a57c8a84103478ba18cab6cb1c064d0', 'request_id': 'req-2113a87e-3ee7-40f2-8c18-8361096b1ccc'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:14.907908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a57c8a84103478ba18cab6cb1c064d0', 'request_id': 'req-2113a87e-3ee7-40f2-8c18-8361096b1ccc'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:14.916347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a57c8a84103478ba18cab6cb1c064d0', 'request_id': 'req-2113a87e-3ee7-40f2-8c18-8361096b1ccc'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.022751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] The token cache is being invalidate because project 6a57c8a84103478ba18cab6cb1c064d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:15.023313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.024140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.031338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 935/1848] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:14 2022] DELETE /identity/v3/projects/6a57c8a84103478ba18cab6cb1c064d0 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:15.033004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:15.034657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:15.034909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 914/1849] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:15.042624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:15.045327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:15.154687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a7c553-82e2-4cbe-a9b6-800e0f6463a5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:15.159976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:15.161595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:15.167019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:15.167019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:15.167019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:15.174269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113a87e-3ee7-40f2-8c18-8361096b1ccc admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:15.193091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:15.194939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:15.195661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:15.195661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] PATH_INFO: `/v3/users/690fc85a490141de87f6755b928f729c` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:15.199121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=690fc85a490141de87f6755b928f729c)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:15.203202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:15.206847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:15.211340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:15.228800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 915/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:15.374781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:15.376718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:15.405399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085b48-dc23-4069-b245-67421e155060 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:15.419091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:15.421911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:15.422246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690fc85a490141de87f6755b928f729c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.422636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:15.423139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:15.423693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690fc85a490141de87f6755b928f729c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.424432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690fc85a490141de87f6755b928f729c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.430881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:15.436857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690fc85a490141de87f6755b928f729c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.522717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b2d6d478-0e51-4b4b-85cb-ae545bbdf094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690fc85a490141de87f6755b928f729c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:15.534036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:15.535872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:15.558515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 916/1851] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:15.571111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 936/1852] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:15 2022] DELETE /identity/v3/users/690fc85a490141de87f6755b928f729c => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:16.017874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:16.238454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:16.240046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:16.256566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091f8c1-8ebe-4d5a-995e-f395e913b8e1 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:16.346929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:16.347582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:16.347865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:16.348173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] PATH_INFO: `/v3/projects/9e23d7e22aab4a08a7dcb51862aff52b` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:16.349358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e23d7e22aab4a08a7dcb51862aff52b)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:16.351329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:16.589967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e23d7e22aab4a08a7dcb51862aff52b', 'request_id': 'req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:16.604756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e23d7e22aab4a08a7dcb51862aff52b', 'request_id': 'req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:16.622018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e23d7e22aab4a08a7dcb51862aff52b', 'request_id': 'req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:16.745966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] The token cache is being invalidate because project 9e23d7e22aab4a08a7dcb51862aff52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:16.746581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:16.748935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-88dc77f9-b4ae-42b2-bedb-d172e80a4d0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:16.751918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 917/1853] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:16 2022] DELETE /identity/v3/projects/9e23d7e22aab4a08a7dcb51862aff52b => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:16.756960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:16.757247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:16.757484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:16.789182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:16.795505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:16.874151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 937/1854] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:16.882325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704781c8-5483-4981-b74d-36535ca56148 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:16.882640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704781c8-5483-4981-b74d-36535ca56148 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:16.882919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704781c8-5483-4981-b74d-36535ca56148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:16.910832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-704781c8-5483-4981-b74d-36535ca56148 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:16.912331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-704781c8-5483-4981-b74d-36535ca56148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:16.921702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 918/1855] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:16.926643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:16.940337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd47b85f-7ddc-45a9-a9a7-bef43b639549 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:16.945686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:16.946311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:16.946588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:16.946889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:16.947852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:16.948088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:16.949942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:16.964051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 938/1856] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:23:16.968246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704781c8-5483-4981-b74d-36535ca56148 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:16.977973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704781c8-5483-4981-b74d-36535ca56148 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:16.983302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:16.983910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:16.984290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:16.984505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:16.985136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:16.987318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.101238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 919/1857] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:16 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:17.102530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.115060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37101343-a838-4908-88e2-1c0e86a3840a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.123417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:17.174601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.175513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.175798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.176097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.177116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.179398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.276258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 939/1858] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:17 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:17.280558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.293702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c0d5f-8cf9-4c3a-90b2-9321264dda2f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.300132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.300739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.301022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.301295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.301944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.303574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.318428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 920/1859] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:17.322570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.334160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6ebfe-2d30-4179-9356-d25b77417455 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.366918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb30c94-4b3a-4353-bab1-0aef23c49cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.367544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30c94-4b3a-4353-bab1-0aef23c49cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.367817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30c94-4b3a-4353-bab1-0aef23c49cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.368171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb30c94-4b3a-4353-bab1-0aef23c49cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.368763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb30c94-4b3a-4353-bab1-0aef23c49cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.370575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb30c94-4b3a-4353-bab1-0aef23c49cfa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.384067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 940/1860] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:17.388778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.400127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb13d43-4235-43fe-b751-9f84b9db28a9 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.407220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.407979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.408730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.409081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] PATH_INFO: `/v3/projects/64b698e3831346bb8064151008b39428/users/e0a8e09fa3f64af8b3ed45858cc38b40/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.427553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b698e3831346bb8064151008b39428, user_id=e0a8e09fa3f64af8b3ed45858cc38b40, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.430638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.563818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 921/1861] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:17 2022] PUT /identity/v3/projects/64b698e3831346bb8064151008b39428/users/e0a8e09fa3f64af8b3ed45858cc38b40/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:17.577869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.578155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.578604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.607283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] MFA Rules not processed for user `e0a8e09fa3f64af8b3ed45858cc38b40`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:17.608673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:17.678791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 941/1862] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:17.683348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.726693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.727751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:17.754996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b2a15-fff4-4c8c-88f4-33191207a716 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.759601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.760338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.760665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.760936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.761624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.764268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.802846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e54905-6d08-4d39-80b9-5f5e1dfdbd89 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.802846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.802846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.802846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.803302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] PATH_INFO: `/v3/users/d28293d25e2b40a4a992c3beffe1faad` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.803302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28293d25e2b40a4a992c3beffe1faad)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.804688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:17.918158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 922/1863] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:17 2022] POST /identity/v3/projects => generated 383 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:17.923078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:17.979029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:17.984546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c57503-687b-463c-b886-daa6876370cd admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:17.989349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:17.992527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:17.992527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:17.992527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:17.992527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:17.993124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.096586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 923/1864] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:17 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:18.104208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28293d25e2b40a4a992c3beffe1faad'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:18.104929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:18.105097 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28293d25e2b40a4a992c3beffe1faad'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:18.105710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28293d25e2b40a4a992c3beffe1faad'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:18.115504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28293d25e2b40a4a992c3beffe1faad'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:18.116571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:18.129879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f88e2e-5ae9-49ab-8a15-377d9799d486 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:18.135575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:18.136320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.136613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.136946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.137540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:18.139752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.150190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 924/1865] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:18.154929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:18.165830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cf59b-f1de-408e-aeb3-3fd94990dac7 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:18.170425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:18.171079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.171353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.172720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] PATH_INFO: `/v3/projects/01e28a58e928453a859c9b74650666c4/users/3f0b31e8627743d2a71d9a695f0cb4a8/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.186768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e28a58e928453a859c9b74650666c4, user_id=3f0b31e8627743d2a71d9a695f0cb4a8, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:18.189449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.195893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4ce30c44-67da-4f8c-afe3-7070b39e510a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28293d25e2b40a4a992c3beffe1faad'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:18.215819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 942/1866] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:17 2022] DELETE /identity/v3/users/d28293d25e2b40a4a992c3beffe1faad => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:18.221142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.221451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.221726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.241186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:18.242471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:18.263941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 925/1867] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:18 2022] PUT /identity/v3/projects/01e28a58e928453a859c9b74650666c4/users/3f0b31e8627743d2a71d9a695f0cb4a8/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:18.268168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:18.317621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 943/1868] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:18.330995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:18.330995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:18.356951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b7716-079c-497b-8091-389b345ca316 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:18.362414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:18.363029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.363313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.363716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.373718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:18.375381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.393506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 926/1869] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:18.398326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:18.401822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d07210-e42d-4779-958c-16e5681f8584 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:18.411249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f23d5-012d-4b56-b0d0-31845c3fb506 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:18.416446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab6e78-5def-4c09-8456-070889ac881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:18.417191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab6e78-5def-4c09-8456-070889ac881a admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.417531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab6e78-5def-4c09-8456-070889ac881a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.420306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab6e78-5def-4c09-8456-070889ac881a admin admin] PATH_INFO: `/v3/projects/01e28a58e928453a859c9b74650666c4/users/3f0b31e8627743d2a71d9a695f0cb4a8/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.420306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab6e78-5def-4c09-8456-070889ac881a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01e28a58e928453a859c9b74650666c4, user_id=3f0b31e8627743d2a71d9a695f0cb4a8, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:18.422814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab6e78-5def-4c09-8456-070889ac881a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.434363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 927/1870] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:18 2022] PUT /identity/v3/projects/01e28a58e928453a859c9b74650666c4/users/3f0b31e8627743d2a71d9a695f0cb4a8/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:18.444594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.444996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.445381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.478153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:18.478971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.479543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.480363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.481920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] MFA Rules not processed for user `3f0b31e8627743d2a71d9a695f0cb4a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:18.483394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:18.492934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:18.495115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.508244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 944/1871] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:18.587625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 928/1872] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:18.601891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:18.604146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:18.673791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02232ed-2cb8-49f1-9d8e-0a11d382f746 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:18.726574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:18.727471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:18.727744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:18.728032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:18.738106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:18.741384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:18.750367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 945/1873] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:19.038756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:19.048648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ba40e-9747-498c-a5a8-7f71dc969b02 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:19.052268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:19.052866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:19.053146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:19.053419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:19.055105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:19.115124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:19.117153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:19.117597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:19.119906 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:19.128228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 929/1874] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:19.196434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0fdb-bcdd-4215-963b-d9686bbab6f4 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:19.201967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:19.202617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:19.202894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:19.203168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] PATH_INFO: `/v3/projects/9c70a7a3fcf6408c9539c913c4b4140e` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:19.204382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c70a7a3fcf6408c9539c913c4b4140e)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:19.206340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:19.316005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c70a7a3fcf6408c9539c913c4b4140e', 'request_id': 'req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.321788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c70a7a3fcf6408c9539c913c4b4140e', 'request_id': 'req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.328657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c70a7a3fcf6408c9539c913c4b4140e', 'request_id': 'req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.481468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] The token cache is being invalidate because project 9c70a7a3fcf6408c9539c913c4b4140e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:19.481468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.482732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.484303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 946/1875] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:19 2022] DELETE /identity/v3/projects/9c70a7a3fcf6408c9539c913c4b4140e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:19.488549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:19.504557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecdcbd-4378-4e65-abd7-8a1061038848 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:19.558594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:19.559376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:19.559759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:19.560331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] PATH_INFO: `/v3/users/c153c898e5804b3392234d1e5b026366` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:19.561575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c153c898e5804b3392234d1e5b026366)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:19.563619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:19.790855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:19.792397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:19.795341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c153c898e5804b3392234d1e5b026366'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.802861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c153c898e5804b3392234d1e5b026366'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.804477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c153c898e5804b3392234d1e5b026366'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.808398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fe0fb-3a6b-4b9b-a6d8-d2338c047303 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:19.813053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c153c898e5804b3392234d1e5b026366'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.814570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:19.815496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:19.815826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:19.816535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] PATH_INFO: `/v3/users/e0a8e09fa3f64af8b3ed45858cc38b40` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:19.817417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a8e09fa3f64af8b3ed45858cc38b40)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:19.819527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:19.945834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d086c8da-2971-49e1-ad47-4c69d85fdc3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c153c898e5804b3392234d1e5b026366'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:19.981292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 930/1876] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:19 2022] DELETE /identity/v3/users/c153c898e5804b3392234d1e5b026366 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:20.101928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a8e09fa3f64af8b3ed45858cc38b40'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.103714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a8e09fa3f64af8b3ed45858cc38b40'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.104563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a8e09fa3f64af8b3ed45858cc38b40'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.113637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a8e09fa3f64af8b3ed45858cc38b40'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.184126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a8e09fa3f64af8b3ed45858cc38b40'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.204628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 947/1877] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:19 2022] DELETE /identity/v3/users/e0a8e09fa3f64af8b3ed45858cc38b40 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:20.209196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:20.211837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:20.212225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:20.231202 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:20.232552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:20.299010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 931/1878] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:20.314173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:20.315634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:20.378545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab87ed-a19e-442d-8185-0be1f89f0896 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:20.384992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fcd92-17da-4d1d-8a1b-c8a74ea03b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:20.385704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fcd92-17da-4d1d-8a1b-c8a74ea03b2c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:20.386307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fcd92-17da-4d1d-8a1b-c8a74ea03b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:20.386307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fcd92-17da-4d1d-8a1b-c8a74ea03b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:20.396356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fcd92-17da-4d1d-8a1b-c8a74ea03b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:20.397797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fcd92-17da-4d1d-8a1b-c8a74ea03b2c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:20.409606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 948/1879] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:20.643674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:20.646188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:20.670398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cdbeb6-e25b-4992-b816-7a3993a56b9f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:20.670398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:20.670844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:20.671642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:20.672244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] PATH_INFO: `/v3/projects/c88f18a655b54ccbb7fbc92cdb4cb567` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:20.673640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c88f18a655b54ccbb7fbc92cdb4cb567)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:20.676032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:20.797270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c88f18a655b54ccbb7fbc92cdb4cb567', 'request_id': 'req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.804355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c88f18a655b54ccbb7fbc92cdb4cb567', 'request_id': 'req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.811301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c88f18a655b54ccbb7fbc92cdb4cb567', 'request_id': 'req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.933224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] The token cache is being invalidate because project c88f18a655b54ccbb7fbc92cdb4cb567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:20.933751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.934575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-004ab9bd-862f-43a8-a36f-e9d8b6ad4c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:20.949122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 932/1880] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:20 2022] DELETE /identity/v3/projects/c88f18a655b54ccbb7fbc92cdb4cb567 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:20.949226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:20.949226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:20.949226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:20.967118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:20.968517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:21.032337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 949/1881] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:21.039963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.040287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.040634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.072943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.074590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:21.075467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:21.078265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:21.086832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 933/1882] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:21.091446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.103732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d23a3-e5bf-4757-831b-2eaab7659520 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.106724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c201ceb9-4a91-49a8-b450-fd7deda88aab None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.111649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.113078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.113597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.113911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.114604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.114882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.115161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.115627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] PATH_INFO: `/v3/projects/64b698e3831346bb8064151008b39428` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.117415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b698e3831346bb8064151008b39428)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.125493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.128249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:21.128744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.130743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.147117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 934/1883] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:23:21.151261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.168763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14674f-995e-481f-a636-857bb4be59f2 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.175473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.176397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.176618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.176861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.177497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.180281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.290218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b698e3831346bb8064151008b39428', 'request_id': 'req-6bf122ab-d3ca-462b-9859-f7ef4122920f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.299240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b698e3831346bb8064151008b39428', 'request_id': 'req-6bf122ab-d3ca-462b-9859-f7ef4122920f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.315135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b698e3831346bb8064151008b39428', 'request_id': 'req-6bf122ab-d3ca-462b-9859-f7ef4122920f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.323740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 935/1884] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:21 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:21.327794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.414760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda2528-17cb-4024-8e44-bc91ee506562 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.420237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.420974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.421256 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.421530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.422154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.424417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.443288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] The token cache is being invalidate because project 64b698e3831346bb8064151008b39428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:21.443288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.443288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.445040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 950/1885] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:21 2022] DELETE /identity/v3/projects/64b698e3831346bb8064151008b39428 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:21.449095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.450876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:21.464945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:21.532443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 936/1886] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:21 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:21.538959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.541131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:21.541262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf122ab-d3ca-462b-9859-f7ef4122920f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.547805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.548476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.549085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.550217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] PATH_INFO: `/v3/users/3f0b31e8627743d2a71d9a695f0cb4a8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.552380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f0b31e8627743d2a71d9a695f0cb4a8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.555587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.560772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67047864-14b0-4d09-9d09-b7d7b4a9db01 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.565922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.566977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.566977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.570096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.570096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.570096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.583954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 937/1887] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:21.585055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.596052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58165f94-2cfc-442c-91c7-a31c02d0de66 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.600810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.601398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.604507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.604507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.604507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.604507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.615263 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 938/1888] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:21.619401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.629814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8400c-1048-4ce1-ae92-07df103cd4bc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.634687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.635340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.635619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.636079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] PATH_INFO: `/v3/projects/dc640c14e65549a19a14c8bfaf447a72/users/1461e7e4182e4be8bef5c6012ffc4d15/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.651329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc640c14e65549a19a14c8bfaf447a72, user_id=1461e7e4182e4be8bef5c6012ffc4d15, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.655500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.831643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 939/1889] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:21 2022] PUT /identity/v3/projects/dc640c14e65549a19a14c8bfaf447a72/users/1461e7e4182e4be8bef5c6012ffc4d15/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:21.832765 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:21.834359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:21.852248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f0b31e8627743d2a71d9a695f0cb4a8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.853160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f0b31e8627743d2a71d9a695f0cb4a8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.853817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f0b31e8627743d2a71d9a695f0cb4a8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.868206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f0b31e8627743d2a71d9a695f0cb4a8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.900523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc717cc0-78fb-4a2e-9bf8-03840c22b65e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:21.906095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:21.906720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.906998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.907271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] PATH_INFO: `/v3/users/c690fa446a8046d498eb41b916cb431e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:21.908427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c690fa446a8046d498eb41b916cb431e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:21.910336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:21.950304 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fe26cad7-bb1d-4e08-af1d-170225f00cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f0b31e8627743d2a71d9a695f0cb4a8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:21.969346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 951/1890] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:21 2022] DELETE /identity/v3/users/3f0b31e8627743d2a71d9a695f0cb4a8 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:21.970572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:21.970855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:21.971131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.008893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] MFA Rules not processed for user `1461e7e4182e4be8bef5c6012ffc4d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:22.010158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.083725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 952/1891] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:22.088168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.089492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.156315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571556d2-89de-4af7-a679-cb47482dd520 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.163358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.164356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.164356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.175780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.175780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.175780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.227205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c690fa446a8046d498eb41b916cb431e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.228080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c690fa446a8046d498eb41b916cb431e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.228956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c690fa446a8046d498eb41b916cb431e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.236727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c690fa446a8046d498eb41b916cb431e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.281178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 953/1892] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:22 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:22.282656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.284401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.327734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-932c9241-17a1-49e7-8435-5a3b6d92ab45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c690fa446a8046d498eb41b916cb431e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.357915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 940/1893] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:21 2022] DELETE /identity/v3/users/c690fa446a8046d498eb41b916cb431e => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:22.362552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.362642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.362913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.363599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431532d1-1c70-4b13-81fd-34e7d49bb56f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.373321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:22.400766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:22.402042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.459389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.460223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.460502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.460865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.461570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.468332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.480028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 941/1894] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:22.480961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.482573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.494586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1995491a-35b8-4eb3-a7cd-07605725afe4 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.499960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.500613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.500888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.501186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] PATH_INFO: `/v3/projects/01e28a58e928453a859c9b74650666c4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.502290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01e28a58e928453a859c9b74650666c4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.504286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.635797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 954/1895] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:22 2022] POST /identity/v3/users => generated 297 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:22.637052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.639169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.682997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01e28a58e928453a859c9b74650666c4', 'request_id': 'req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.692106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01e28a58e928453a859c9b74650666c4', 'request_id': 'req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.698802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01e28a58e928453a859c9b74650666c4', 'request_id': 'req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.701252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e810e93-9942-453e-9985-957520dbf9e9 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.751540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.752254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.752530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.752797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.811119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.812588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.821834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 955/1896] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:22.822630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.832198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b228a7e-e43a-4d1b-b7b1-60b8c69554dc service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.835599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] The token cache is being invalidate because project 01e28a58e928453a859c9b74650666c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:22.835889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.836740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d696e5e3-c9b4-4666-b1b8-2b992a6d84e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:22.836909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.840202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.840202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.840202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.840545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 942/1897] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:22 2022] DELETE /identity/v3/projects/01e28a58e928453a859c9b74650666c4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:22.840635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.840635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.845109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166276b-061f-45f2-880d-dd2ac5e61e8c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.845603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166276b-061f-45f2-880d-dd2ac5e61e8c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.846030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166276b-061f-45f2-880d-dd2ac5e61e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.854663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 956/1898] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:22.855684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.858025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.871131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0fbff5-95ae-4c7c-a434-25cadda77ac2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.873941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a166276b-061f-45f2-880d-dd2ac5e61e8c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:22.875690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a166276b-061f-45f2-880d-dd2ac5e61e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.877059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.877493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.877773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.895175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] PATH_INFO: `/v3/projects/db67d8d13aca4d5f8de311e9c7496f9d/users/12fe56b1287348c9964839f9ee49bd4d/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.895405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 943/1899] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:22.895483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db67d8d13aca4d5f8de311e9c7496f9d, user_id=12fe56b1287348c9964839f9ee49bd4d, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.897058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.903688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.903817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.904494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.925954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:22.927248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:22.935922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 944/1900] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:22.940122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.949372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cab3ef-29af-4592-9359-7f0c5dbfb592 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.953952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:22.954548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:22.954824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:22.955094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:22.955755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:22.956085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:22.957661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:22.969350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 945/1901] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:23:22.974108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:22.980237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 957/1902] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:22 2022] PUT /identity/v3/projects/db67d8d13aca4d5f8de311e9c7496f9d/users/12fe56b1287348c9964839f9ee49bd4d/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:22.984639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847eba8-757d-4d16-a770-854d6250d93b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:22.985488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.052268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.052943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.053231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.053505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.054140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.056211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.109164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f5486-751f-49e1-bc15-168703f313c6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.114176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.114778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.115051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.115318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.115951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.117581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.127702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 958/1903] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:23.131672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.141479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0ef4f7-efca-4625-a784-82a8e37c549b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.146449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c67f0-6f38-4126-a143-35746d65a8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.147176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c67f0-6f38-4126-a143-35746d65a8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.147520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c67f0-6f38-4126-a143-35746d65a8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.147861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c67f0-6f38-4126-a143-35746d65a8e9 admin admin] PATH_INFO: `/v3/projects/db67d8d13aca4d5f8de311e9c7496f9d/users/12fe56b1287348c9964839f9ee49bd4d/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.150010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c67f0-6f38-4126-a143-35746d65a8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db67d8d13aca4d5f8de311e9c7496f9d, user_id=12fe56b1287348c9964839f9ee49bd4d, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.152733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509c67f0-6f38-4126-a143-35746d65a8e9 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.165134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 959/1904] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:23 2022] PUT /identity/v3/projects/db67d8d13aca4d5f8de311e9c7496f9d/users/12fe56b1287348c9964839f9ee49bd4d/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:23.181675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.181948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.182231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.210612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 946/1905] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:22 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:23.213019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.222487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] MFA Rules not processed for user `12fe56b1287348c9964839f9ee49bd4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:23.223832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:23.288047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478628-2994-4f15-91e7-6527caa37e34 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.293038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.293674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.293951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.294806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.295463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.297358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.301216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 960/1906] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:23.312932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.314347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:23.374636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a8216-bbe7-4777-8f86-0c3ccc39da22 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.379111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.385453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.385453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.385453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.406018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.406018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.412288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 961/1907] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:23.413059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.415415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:23.431819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92df9fd-ad3e-4cb3-8cc4-c8db37f16eef service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.432919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 947/1908] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:23 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:23.437972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.441375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.442148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.442498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.442842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] PATH_INFO: `/v3/projects/2941567e527a4cf9974002c690191d1a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.444722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2941567e527a4cf9974002c690191d1a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.448272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.474153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a29718-1548-4117-9498-67d0ee6139ea admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.480403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.481225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.482532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.482986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.484301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.486699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.499704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 948/1909] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:23.503780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.515222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b46f19-00e2-442b-aa46-0172d3a3a052 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.520208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.520812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.521083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.521367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.522128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.523901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.589607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 949/1910] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:23.600348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.600795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2941567e527a4cf9974002c690191d1a', 'request_id': 'req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:23.607639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2941567e527a4cf9974002c690191d1a', 'request_id': 'req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:23.615253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2941567e527a4cf9974002c690191d1a', 'request_id': 'req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:23.616511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c76a8b-c383-42d1-aa99-e8fa8cca0b35 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.682608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98b9fa0-743d-4b22-8c90-4002b45a43aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.683359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b9fa0-743d-4b22-8c90-4002b45a43aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.684265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b9fa0-743d-4b22-8c90-4002b45a43aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.684682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98b9fa0-743d-4b22-8c90-4002b45a43aa admin admin] PATH_INFO: `/v3/projects/79f14eb6c6204e17a3849f929cc48be7/users/5f85f1ffac494e8580194b30367e2cd4/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.712694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] The token cache is being invalidate because project 2941567e527a4cf9974002c690191d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:23.712694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:23.713160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b9fa0-743d-4b22-8c90-4002b45a43aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=79f14eb6c6204e17a3849f929cc48be7, user_id=5f85f1ffac494e8580194b30367e2cd4, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.713965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:23.717404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 962/1911] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:23 2022] DELETE /identity/v3/projects/2941567e527a4cf9974002c690191d1a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:23.722455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98b9fa0-743d-4b22-8c90-4002b45a43aa admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.730237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:23.730237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:23.751559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1bb057-30bb-4ce6-976c-149f0ad75ce5 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:23.762058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:23.763660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.763870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.764282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] PATH_INFO: `/v3/users/738dcba7096b44d492653def32c2ea6d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.765593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=738dcba7096b44d492653def32c2ea6d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:23.779926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:23.894352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 950/1912] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:23 2022] PUT /identity/v3/projects/79f14eb6c6204e17a3849f929cc48be7/users/5f85f1ffac494e8580194b30367e2cd4/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:23.904800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:23.905087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:23.905370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:23.939346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] MFA Rules not processed for user `5f85f1ffac494e8580194b30367e2cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:23.940624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.012900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:24.019094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 951/1913] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:24.026331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:24.027673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.061839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '738dcba7096b44d492653def32c2ea6d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.062532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '738dcba7096b44d492653def32c2ea6d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.063233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '738dcba7096b44d492653def32c2ea6d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.070437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '738dcba7096b44d492653def32c2ea6d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.084826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c9284-7c27-412d-91b5-1a23205a5c21 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:24.088392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:24.090847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:24.090847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:24.090847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:24.091411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.102724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:24.104420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:24.113907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 952/1914] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:24.114904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:24.116259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.172856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '738dcba7096b44d492653def32c2ea6d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.199297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 963/1915] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:23 2022] DELETE /identity/v3/users/738dcba7096b44d492653def32c2ea6d => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:24.204223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee16901-bd77-40f1-a697-e5b1aa3b5b40 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:24.283738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:24.284636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:24.285178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:24.285609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] PATH_INFO: `/v3/users/1461e7e4182e4be8bef5c6012ffc4d15` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:24.287051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1461e7e4182e4be8bef5c6012ffc4d15)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:24.289764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:24.626325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1461e7e4182e4be8bef5c6012ffc4d15'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.626978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1461e7e4182e4be8bef5c6012ffc4d15'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.627586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1461e7e4182e4be8bef5c6012ffc4d15'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.634179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1461e7e4182e4be8bef5c6012ffc4d15'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.676487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:24.677084 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.692284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6034b6-a950-4f6a-9899-547c1d941dc4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:24.705889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:24.706517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:24.706801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:24.707067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] PATH_INFO: `/v3/projects/64d9ea8b5e4f47eeaa007f5268ceaf7f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:24.708265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d9ea8b5e4f47eeaa007f5268ceaf7f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:24.710687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:24.761105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5dc91533-4aa4-4179-8b7f-0f526b7b0fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1461e7e4182e4be8bef5c6012ffc4d15'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.781025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 953/1916] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:24 2022] DELETE /identity/v3/users/1461e7e4182e4be8bef5c6012ffc4d15 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:24.785313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:24.785601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:24.785912 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:24.807155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:24.808459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.871720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 954/1917] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:24.882640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d9ea8b5e4f47eeaa007f5268ceaf7f', 'request_id': 'req-a183ba48-309e-4ff2-b30d-04161d492d09'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.888767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d9ea8b5e4f47eeaa007f5268ceaf7f', 'request_id': 'req-a183ba48-309e-4ff2-b30d-04161d492d09'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.896618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d9ea8b5e4f47eeaa007f5268ceaf7f', 'request_id': 'req-a183ba48-309e-4ff2-b30d-04161d492d09'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:24.905096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:24.906552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:24.982216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92550fb1-e6ce-430b-9424-248a583dbfcf None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:25.032357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:25.032892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:25.033159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:25.033427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:25.077106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] The token cache is being invalidate because project 64d9ea8b5e4f47eeaa007f5268ceaf7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:25.078006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.079186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.081218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 964/1918] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:24 2022] DELETE /identity/v3/projects/64d9ea8b5e4f47eeaa007f5268ceaf7f => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:25.084908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:25.087235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:25.103481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:25.104989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:25.116545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 955/1919] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:25.163610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183ba48-309e-4ff2-b30d-04161d492d09 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:25.168827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:25.170066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:25.170354 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:25.170671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] PATH_INFO: `/v3/users/0ddf1eaa49fa4708873106eddd11b178` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:25.171891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ddf1eaa49fa4708873106eddd11b178)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:25.173911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:25.382491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ddf1eaa49fa4708873106eddd11b178'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.383257 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ddf1eaa49fa4708873106eddd11b178'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.383876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ddf1eaa49fa4708873106eddd11b178'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.393019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ddf1eaa49fa4708873106eddd11b178'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.498987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ddf1eaa49fa4708873106eddd11b178'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:25.521767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 965/1920] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:25 2022] DELETE /identity/v3/users/0ddf1eaa49fa4708873106eddd11b178 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:25.734323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:25.941109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:25.942624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:25.955566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbb3ef-7d6e-428f-9326-920ce92599f5 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:25.967760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:26.032931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:26.033875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.034594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.035380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] PATH_INFO: `/v3/projects/dc640c14e65549a19a14c8bfaf447a72` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.037413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc640c14e65549a19a14c8bfaf447a72)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:26.039576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:26.212310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:26.214209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:26.222835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc640c14e65549a19a14c8bfaf447a72', 'request_id': 'req-4bee91e9-3b33-44bd-968a-f32f64ef1522'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.230087 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316aba5-f6e7-45f1-aa7c-f7325e257f25 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:26.230814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc640c14e65549a19a14c8bfaf447a72', 'request_id': 'req-4bee91e9-3b33-44bd-968a-f32f64ef1522'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.237369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc640c14e65549a19a14c8bfaf447a72', 'request_id': 'req-4bee91e9-3b33-44bd-968a-f32f64ef1522'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.293494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:26.294186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.294790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.294790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] PATH_INFO: `/v3/projects/cf24b8ca948f4d8b93020371e7e5eff2` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.295903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf24b8ca948f4d8b93020371e7e5eff2)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:26.297913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:26.344936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] The token cache is being invalidate because project dc640c14e65549a19a14c8bfaf447a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:26.345261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.345915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.347424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 956/1921] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:25 2022] DELETE /identity/v3/projects/dc640c14e65549a19a14c8bfaf447a72 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:26.351286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:26.352761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:26.365306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee91e9-3b33-44bd-968a-f32f64ef1522 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:26.376108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:26.376779 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.377306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.377586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] PATH_INFO: `/v3/users/12fe56b1287348c9964839f9ee49bd4d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.378850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12fe56b1287348c9964839f9ee49bd4d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:26.386105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:26.457950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf24b8ca948f4d8b93020371e7e5eff2', 'request_id': 'req-358417b5-77d2-405f-a718-bbbf0a4cc84c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.463894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf24b8ca948f4d8b93020371e7e5eff2', 'request_id': 'req-358417b5-77d2-405f-a718-bbbf0a4cc84c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.470931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf24b8ca948f4d8b93020371e7e5eff2', 'request_id': 'req-358417b5-77d2-405f-a718-bbbf0a4cc84c'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.608321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] The token cache is being invalidate because project cf24b8ca948f4d8b93020371e7e5eff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:26.608657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.609365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-358417b5-77d2-405f-a718-bbbf0a4cc84c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.610885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 966/1922] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:26 2022] DELETE /identity/v3/projects/cf24b8ca948f4d8b93020371e7e5eff2 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:26.616525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1dd77-2a25-4ba2-982f-405a7a13594f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.617051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1dd77-2a25-4ba2-982f-405a7a13594f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.617670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f1dd77-2a25-4ba2-982f-405a7a13594f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.639166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d6f1dd77-2a25-4ba2-982f-405a7a13594f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:26.640423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d6f1dd77-2a25-4ba2-982f-405a7a13594f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:26.695335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12fe56b1287348c9964839f9ee49bd4d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.696048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12fe56b1287348c9964839f9ee49bd4d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.696673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12fe56b1287348c9964839f9ee49bd4d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.704316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12fe56b1287348c9964839f9ee49bd4d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.705815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 967/1923] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:26.714131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.714418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.714695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.734044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:26.735233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:26.744054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 968/1924] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:26.748823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:26.757939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a53849-8925-4b61-a954-bd6f2019b184 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:26.762674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:26.778004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.778004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.778004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.778004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:26.778004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:26.778004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:26.778496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 969/1925] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:26.782604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:26.792445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ccaa0-4ebd-43e4-b9af-a35c0cfe8895 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:26.798010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:26.798654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:26.798935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:26.799211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:26.799840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:26.802882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:26.803973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12fe56b1287348c9964839f9ee49bd4d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:26.828835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 957/1926] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:26 2022] DELETE /identity/v3/users/12fe56b1287348c9964839f9ee49bd4d => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:26.917046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 970/1927] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:26 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:26.921919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:26.986718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061ed08-f63c-4d41-abcb-606fa775d463 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.041507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.042151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.042434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.042709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.043472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.046009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.178322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 958/1928] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:26 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:27.182585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.195262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3595f4-b1e9-407f-9916-6b4221379f9f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.215678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.219180 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.222810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.222810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.222810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.222810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.254534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 971/1929] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:27.258582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.271771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7ef48-28c1-44dc-9c5d-ffabab285e41 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.277339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.278008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.278295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.279274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.280086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.282001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.293650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 959/1930] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:27.298043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.309025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea43683-0cb7-4d83-8e88-139f7028d9b5 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.315915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257fa9b-febb-42b3-a3d8-69966dd515f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.316719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fa9b-febb-42b3-a3d8-69966dd515f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.317066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fa9b-febb-42b3-a3d8-69966dd515f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.317405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257fa9b-febb-42b3-a3d8-69966dd515f1 admin admin] PATH_INFO: `/v3/projects/b38b24a11e514fd78c4649fbf4643a3c/users/26f8415e4ecc4f7e9ad866713b58fbce/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.340498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257fa9b-febb-42b3-a3d8-69966dd515f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38b24a11e514fd78c4649fbf4643a3c, user_id=26f8415e4ecc4f7e9ad866713b58fbce, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.344307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e257fa9b-febb-42b3-a3d8-69966dd515f1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.388816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.391623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:27.415001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb24ad-30ec-4671-8400-83c031e850ad admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.422517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.423383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.423723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.424171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] PATH_INFO: `/v3/projects/db67d8d13aca4d5f8de311e9c7496f9d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.425643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=db67d8d13aca4d5f8de311e9c7496f9d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.427868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.435140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 972/1931] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:27 2022] PUT /identity/v3/projects/b38b24a11e514fd78c4649fbf4643a3c/users/26f8415e4ecc4f7e9ad866713b58fbce/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:27.449667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.451118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.451412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.505997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] MFA Rules not processed for user `26f8415e4ecc4f7e9ad866713b58fbce`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:27.507421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:27.608347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 973/1932] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:27.618311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.618517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db67d8d13aca4d5f8de311e9c7496f9d', 'request_id': 'req-0500da71-2f1b-4e9f-8e2d-766455829c6e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:27.620811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:27.626950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db67d8d13aca4d5f8de311e9c7496f9d', 'request_id': 'req-0500da71-2f1b-4e9f-8e2d-766455829c6e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:27.633039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db67d8d13aca4d5f8de311e9c7496f9d', 'request_id': 'req-0500da71-2f1b-4e9f-8e2d-766455829c6e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:27.706436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dbd127-b4e7-4e57-8c88-06fd62f62f7b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.718702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:27.722505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] The token cache is being invalidate because project db67d8d13aca4d5f8de311e9c7496f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:27.722732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:27.729628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0500da71-2f1b-4e9f-8e2d-766455829c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:27.731489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 960/1933] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:27 2022] DELETE /identity/v3/projects/db67d8d13aca4d5f8de311e9c7496f9d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:27.738330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ae871-134d-4ec8-81b5-4eaa452762b3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.738608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ae871-134d-4ec8-81b5-4eaa452762b3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.738877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ae871-134d-4ec8-81b5-4eaa452762b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.774902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-788ae871-134d-4ec8-81b5-4eaa452762b3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:27.776593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-788ae871-134d-4ec8-81b5-4eaa452762b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:27.795528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.795841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.796175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.796445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.799340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:27.856149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 961/1934] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:27.864233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.864656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.864936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.867234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.868747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.877941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 974/1935] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:27.885389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:27.886600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:27.895095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 962/1936] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:27.899569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.910803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b178c-83b1-46fd-adc7-5c74d358b47f service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.915575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.916206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.916481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.916748 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.917729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:27.918038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.920321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:27.935153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 975/1937] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:27.937571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:27.946690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e601f-8e62-451c-aac1-28682cbb5744 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:27.952404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:27.953012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:27.953287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:27.953559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:27.954203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:27.956118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:28.064177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 963/1938] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:27 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:28.068383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:28.079557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdc0c3-a74d-4a59-91b3-0e6aeac60dbf admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:28.142962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:28.143565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:28.143844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:28.144144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:28.144761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:28.146666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:28.294104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 976/1939] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:28 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:28.298433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:28.313958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd755562-07e4-4d94-af26-52306b6f03c1 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:28.339439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:28.340995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:28.341441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:28.341863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:28.342697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:28.346861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:28.362339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 964/1940] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:28.370058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:28.385422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7f7a3d-be11-4efc-a59a-a9fa0e6591e8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:28.391663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:28.392128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:28.392409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:28.392679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:28.393516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:28.395651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:28.414718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 977/1941] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:28.419385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:28.431822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc4b43-787f-422d-9c6f-d45534b04b49 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:28.447416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:28.457920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c4b086-081f-4a4d-9891-eca17afec249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:28.458640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4b086-081f-4a4d-9891-eca17afec249 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:28.462018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4b086-081f-4a4d-9891-eca17afec249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:28.462018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c4b086-081f-4a4d-9891-eca17afec249 admin admin] PATH_INFO: `/v3/projects/e7ab78a6cf674ee8b4248a773c44dcf4/users/f93839f7aec347b6a3004537dcfebf2d/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:28.495794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4b086-081f-4a4d-9891-eca17afec249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ab78a6cf674ee8b4248a773c44dcf4, user_id=f93839f7aec347b6a3004537dcfebf2d, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:28.698009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c4b086-081f-4a4d-9891-eca17afec249 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:28.702543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:28.704437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:28.717640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd59825-e3f8-4c8f-9921-191e51b18a25 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:28.722136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:28.722774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:28.723048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:28.723314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] PATH_INFO: `/v3/users/5f85f1ffac494e8580194b30367e2cd4` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:28.724437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f85f1ffac494e8580194b30367e2cd4)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:28.726310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:28.991687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 965/1942] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:28 2022] PUT /identity/v3/projects/e7ab78a6cf674ee8b4248a773c44dcf4/users/f93839f7aec347b6a3004537dcfebf2d/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:29.002100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.002388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.002662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.030410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] MFA Rules not processed for user `f93839f7aec347b6a3004537dcfebf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:29.060720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:29.099617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 966/1943] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:29.105145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:29.172637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fae19c-58a6-4633-8daa-c6dcf770afc5 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:29.177184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:29.177794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.178069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.178337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.178960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:29.180909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:29.471495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f85f1ffac494e8580194b30367e2cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:29.472455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f85f1ffac494e8580194b30367e2cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:29.473086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f85f1ffac494e8580194b30367e2cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:29.482009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f85f1ffac494e8580194b30367e2cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:29.487296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 967/1944] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:29 2022] POST /identity/v3/projects => generated 385 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:29.491492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:29.493668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:29.572348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66fdd6-6e67-42ac-acaa-a45b832736f9 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:29.574315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:29.575163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.576624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.576624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.577261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:29.579253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:29.588760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-310be461-5f04-48a5-8402-5071847ae3e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f85f1ffac494e8580194b30367e2cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:29.619446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 978/1945] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:28 2022] DELETE /identity/v3/users/5f85f1ffac494e8580194b30367e2cd4 => generated 0 bytes in 917 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:29.623761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.624082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.624356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.645786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:29.645786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:29.704766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 979/1946] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:29.709601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 968/1947] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:29 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:29.713640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:29.722321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:29.724199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500ed93-3c04-47a1-bb4e-e0377d402507 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:29.728942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:29.729614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.729894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.730163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.730807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:29.732612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:29.733287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:29.743689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 980/1948] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:29.748713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:29.760393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d8bcb-2264-4ef6-b0c5-807d5046d060 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:29.767030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:29.767030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.767030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.767030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] PATH_INFO: `/v3/projects/72fbe5f3e01443478011e69c4d845daf/users/4e3dfd0ce4b34291896edb1cdb48ceff/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.791352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fbe5f3e01443478011e69c4d845daf, user_id=4e3dfd0ce4b34291896edb1cdb48ceff, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:29.795015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:29.808757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8831e-2c6a-40d3-a18f-61acd760b3ef admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:29.881197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:29.882198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:29.882689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:29.883060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:29.897209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:29.899065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:29.916128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 969/1949] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:29.917454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 981/1950] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:29 2022] PUT /identity/v3/projects/72fbe5f3e01443478011e69c4d845daf/users/4e3dfd0ce4b34291896edb1cdb48ceff/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:29.922512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:29.933584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0b518-218c-4328-8421-3c0186f6a45e service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:30.018614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ee3637-fa9e-406f-b4a8-aa1493f9f565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:30.019387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee3637-fa9e-406f-b4a8-aa1493f9f565 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:30.019768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee3637-fa9e-406f-b4a8-aa1493f9f565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:30.022115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ee3637-fa9e-406f-b4a8-aa1493f9f565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:30.022946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee3637-fa9e-406f-b4a8-aa1493f9f565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:30.024853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ee3637-fa9e-406f-b4a8-aa1493f9f565 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:30.040489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 970/1951] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:29 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:30.046179 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:30.060547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73f727-545c-40a0-8825-7a811c3c32b5 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:30.065648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:30.067173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:30.067173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:30.067173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] PATH_INFO: `/v3/projects/72fbe5f3e01443478011e69c4d845daf/users/4e3dfd0ce4b34291896edb1cdb48ceff/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:30.069812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72fbe5f3e01443478011e69c4d845daf, user_id=4e3dfd0ce4b34291896edb1cdb48ceff, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:30.072747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:30.086643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 982/1952] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:30 2022] PUT /identity/v3/projects/72fbe5f3e01443478011e69c4d845daf/users/4e3dfd0ce4b34291896edb1cdb48ceff/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:30.098695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:30.099232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:30.099507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:30.140018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] MFA Rules not processed for user `4e3dfd0ce4b34291896edb1cdb48ceff`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:30.141700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:30.221187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 971/1953] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:30.229437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:30.233543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:30.286634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:30.306062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f483f6-8813-4535-9bec-d103d9766086 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:30.310988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:30.312186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:30.312474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:30.312749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:30.314594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:30.397268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:30.400891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca92ca-8f06-4d74-9618-db88dc5b001c service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:30.412865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 983/1954] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:30.549734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:30.551883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:30.655738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3d6a0-8dd0-4281-9705-4f486af4ed42 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:30.655738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:30.660752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:30.661225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:30.661709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] PATH_INFO: `/v3/projects/79f14eb6c6204e17a3849f929cc48be7` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:30.663167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79f14eb6c6204e17a3849f929cc48be7)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:30.666178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:30.829216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79f14eb6c6204e17a3849f929cc48be7', 'request_id': 'req-a4375809-3a4c-4b77-b8f3-0662d58f14d5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:30.840038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79f14eb6c6204e17a3849f929cc48be7', 'request_id': 'req-a4375809-3a4c-4b77-b8f3-0662d58f14d5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:30.848420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79f14eb6c6204e17a3849f929cc48be7', 'request_id': 'req-a4375809-3a4c-4b77-b8f3-0662d58f14d5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:30.968961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] The token cache is being invalidate because project 79f14eb6c6204e17a3849f929cc48be7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:30.969284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:30.970413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a4375809-3a4c-4b77-b8f3-0662d58f14d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:30.972881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 972/1955] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:30 2022] DELETE /identity/v3/projects/79f14eb6c6204e17a3849f929cc48be7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:30.979955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:30.980480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:30.980480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.003309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:31.004481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:31.091556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 984/1956] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:31.099249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.099604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.099950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.124226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:31.125522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:31.134167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 973/1957] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:31.138938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.150563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e866549d-99ab-4762-a00b-572ad3702207 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.156401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.157026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.157299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.157570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.158244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:31.158540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.160448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.177361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 985/1958] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:31.181513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.191534 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a945431-5802-4121-8d68-97cca2cbbd2f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.196633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.197301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.201181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.201181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.201181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.201998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.296266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 974/1959] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:31 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:31.297920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.308816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5763338-47e4-4886-9d22-c5ffca5ee02a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.422820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.423656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.424254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.424602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.425317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.427447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.602161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 986/1960] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:31 2022] POST /identity/v3/users => generated 286 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:31.606371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.618012 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286413a4-a798-4049-907d-e5cd2ec75602 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.623725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.624369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.624657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.624931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.625584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.627445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.646663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 975/1961] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:31.650509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.670291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e90258-60f2-4484-a0f0-7c95ebe0394e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.676063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44726e2-1dbb-469e-a153-c921d9770082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.676862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44726e2-1dbb-469e-a153-c921d9770082 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.677275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44726e2-1dbb-469e-a153-c921d9770082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.677668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44726e2-1dbb-469e-a153-c921d9770082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.678423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44726e2-1dbb-469e-a153-c921d9770082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.680397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44726e2-1dbb-469e-a153-c921d9770082 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.695485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 987/1962] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:31.700631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.713224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fae21c-c56a-4a2e-a91d-ee01453cffd8 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:31.718906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:31.719750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.720143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.720487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] PATH_INFO: `/v3/projects/b86b9874f91c4a16b567e035eb966362/users/c51993c317d347b0939140d2f520827a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.741859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b86b9874f91c4a16b567e035eb966362, user_id=c51993c317d347b0939140d2f520827a, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:31.745084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:31.845602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 976/1963] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:31 2022] PUT /identity/v3/projects/b86b9874f91c4a16b567e035eb966362/users/c51993c317d347b0939140d2f520827a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:31.857353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:31.857635 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:31.858080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:31.871677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:31.886612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] MFA Rules not processed for user `c51993c317d347b0939140d2f520827a`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:31.887856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:31.962569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 988/1964] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:31.971905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:31.974119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:32.067981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3ebb80-ba64-486d-a968-f8211f1fd776 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:32.071660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff299a1e-9d66-4ee5-8d5f-68393ae26d3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:32.072369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff299a1e-9d66-4ee5-8d5f-68393ae26d3c service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:32.072647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff299a1e-9d66-4ee5-8d5f-68393ae26d3c service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:32.072918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff299a1e-9d66-4ee5-8d5f-68393ae26d3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:32.081848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff299a1e-9d66-4ee5-8d5f-68393ae26d3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:32.083334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff299a1e-9d66-4ee5-8d5f-68393ae26d3c service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:32.093897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 977/1965] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:32.691433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:33.595905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:34.307309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:34.308913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:34.405154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c226a451-bf4c-40ee-96df-4441555d4b90 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:34.415834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:34.415834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:34.415834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:34.415834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] PATH_INFO: `/v3/users/26f8415e4ecc4f7e9ad866713b58fbce` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:34.420783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26f8415e4ecc4f7e9ad866713b58fbce)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:34.423093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:34.556418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:34.646585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26f8415e4ecc4f7e9ad866713b58fbce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:34.647291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26f8415e4ecc4f7e9ad866713b58fbce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:34.647934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26f8415e4ecc4f7e9ad866713b58fbce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:34.669042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26f8415e4ecc4f7e9ad866713b58fbce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:34.756164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26f8415e4ecc4f7e9ad866713b58fbce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:34.784486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 989/1966] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:34 2022] DELETE /identity/v3/users/26f8415e4ecc4f7e9ad866713b58fbce => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:34.790798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:34.797156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:34.798563 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:34.827188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:34.829029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:34.907432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 978/1967] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:34.917962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:34.919927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:34.996888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d4090-6116-4884-a337-6c344c57f125 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:35.003320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d0371-ca1a-4078-9bea-9e58e199b82d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:35.004036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0371-ca1a-4078-9bea-9e58e199b82d service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:35.004437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0371-ca1a-4078-9bea-9e58e199b82d service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:35.004834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d0371-ca1a-4078-9bea-9e58e199b82d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:35.013825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:35.030743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d0371-ca1a-4078-9bea-9e58e199b82d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:35.032397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629d0371-ca1a-4078-9bea-9e58e199b82d service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:35.042719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 990/1968] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:35.907100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:35.911307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:35.926855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c76ad-3331-407c-b906-da7a1da18c8c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:35.932843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:35.936503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:35.936503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:35.936503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] PATH_INFO: `/v3/projects/b38b24a11e514fd78c4649fbf4643a3c` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:35.937727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38b24a11e514fd78c4649fbf4643a3c)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:35.941001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:36.072080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38b24a11e514fd78c4649fbf4643a3c', 'request_id': 'req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:36.083131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38b24a11e514fd78c4649fbf4643a3c', 'request_id': 'req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:36.092906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38b24a11e514fd78c4649fbf4643a3c', 'request_id': 'req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:36.182812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] The token cache is being invalidate because project b38b24a11e514fd78c4649fbf4643a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:36.183115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:36.183801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-65a8c6ce-e8d7-4009-b6cb-f5b27c4c4db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:36.185947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 979/1969] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:35 2022] DELETE /identity/v3/projects/b38b24a11e514fd78c4649fbf4643a3c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:36.191890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.192476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.192476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.215350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:36.217233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:36.256566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:36.289772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 991/1970] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:36.295812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.296255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.297060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.325598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:36.326862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:36.336589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 980/1971] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:36.341147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.352068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dad5a7-6043-4b7a-a40d-6bba5dc3d2e3 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.357663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.358296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.358577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.358844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.359496 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:36.359799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:36.361501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:36.378861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 992/1972] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:36.383378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.392825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8deff-5531-48d8-8df0-ebdba8cfe8bc None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.398554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.399168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.399441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.399712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.400459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:36.403577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:36.432219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:36.556839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 981/1973] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:36 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:36.561671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.583869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979feb0-9da9-4055-b450-c5355d802ce0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.658224 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.658852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.661376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.661695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.662350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:36.671436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:36.848080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 993/1974] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:36 2022] POST /identity/v3/users => generated 288 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:36.852321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.863968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516ad2b-af14-4791-9669-bca2b29faab0 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.868977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.870048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.870048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.870700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.871160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:36.873089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:36.885005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 982/1975] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:36.889098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.902776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85dc6b-cd6f-4fdc-849c-4ec017c4c969 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.909078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f843e8-a885-4169-ab23-c300cbe72e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.909778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f843e8-a885-4169-ab23-c300cbe72e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.910133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f843e8-a885-4169-ab23-c300cbe72e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.910480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f843e8-a885-4169-ab23-c300cbe72e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.911198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f843e8-a885-4169-ab23-c300cbe72e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:36.912941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f843e8-a885-4169-ab23-c300cbe72e85 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:36.929889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 994/1976] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:36.931091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:36.945764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b371-7253-4cab-ba2c-2adc0ebba64a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:36.952726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:36.954827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:36.954827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:36.954827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] PATH_INFO: `/v3/projects/5d18ca74104d4cbb93e2274ef088b2c3/users/176eb10729624b83a330fa7cbd6e4d18/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:36.970882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d18ca74104d4cbb93e2274ef088b2c3, user_id=176eb10729624b83a330fa7cbd6e4d18, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:36.973610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:37.086734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 983/1977] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:36 2022] PUT /identity/v3/projects/5d18ca74104d4cbb93e2274ef088b2c3/users/176eb10729624b83a330fa7cbd6e4d18/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:37.095395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:37.095684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:37.095961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:37.140446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] MFA Rules not processed for user `176eb10729624b83a330fa7cbd6e4d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:37.142963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:37.269810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 995/1978] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:37.277925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:37.365809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe89be3-6f85-4be4-af01-64539abe694b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:37.378613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:37.379479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:37.380024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:37.380518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:37.381360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:37.383702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:37.498258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 984/1979] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:37 2022] POST /identity/v3/projects => generated 375 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:37.502157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:37.523271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae8dd1-b265-46d3-9aab-05ef4418c248 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:37.597944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:37.598659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:37.598937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:37.599210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:37.600293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:37.603032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:37.803555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 996/1980] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:37 2022] POST /identity/v3/users => generated 288 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:37.804090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:37.834082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b796747-7231-4ecf-b1ad-7a90b2011048 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:37.840496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:37.843777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:37.844196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:37.844579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:37.845780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:37.849626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:37.870088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 985/1981] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:37.871815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:37.886344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eebc405-2f85-4a62-bb04-81a20c815d99 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:37.891893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36d66b-5cfa-4e5c-a23d-a08914a60bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:37.892640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36d66b-5cfa-4e5c-a23d-a08914a60bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:37.892993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36d66b-5cfa-4e5c-a23d-a08914a60bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:37.893340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36d66b-5cfa-4e5c-a23d-a08914a60bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:37.894066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36d66b-5cfa-4e5c-a23d-a08914a60bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:37.899302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36d66b-5cfa-4e5c-a23d-a08914a60bb5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:37.922688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:37.923050 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 997/1982] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:37.943177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caef64c-c614-41e9-9b00-a7cf8eb0fe8a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:37.949796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:37.951154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:37.951648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:37.952132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] PATH_INFO: `/v3/projects/19b08fde07e942ada3316e7c3e31af52/users/cea04a2f4514429a8ef07ea0a03f15f9/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:37.973482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:38.001095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b08fde07e942ada3316e7c3e31af52, user_id=cea04a2f4514429a8ef07ea0a03f15f9, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:38.014347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:38.114174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:38.133351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:38.133638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:38.133920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:38.136981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 986/1983] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:37 2022] PUT /identity/v3/projects/19b08fde07e942ada3316e7c3e31af52/users/cea04a2f4514429a8ef07ea0a03f15f9/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:38.170847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] MFA Rules not processed for user `cea04a2f4514429a8ef07ea0a03f15f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:38.172807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:38.269362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 998/1984] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:38.273732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:38.359727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3461f1a-9ce2-4b39-a127-f4e3e529dc44 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:38.370101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:38.370101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:38.370101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:38.370101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:38.370510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:38.370510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:38.494081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 987/1985] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:38 2022] POST /identity/v3/projects => generated 375 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:38.495333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:38.510994 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed92ac6-e977-4d97-ab44-0185857ae941 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:38.587612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:38.588360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:38.588700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:38.588928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:38.590101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:38.592859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:38.716865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:38.729218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 999/1986] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:38 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:38.734415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:38.758559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edff2b9-bb0a-42a0-9d12-515372beab62 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:38.766383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:38.767384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:38.767861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:38.768351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:38.769147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:38.771351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:38.785261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 988/1987] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:38.789630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:38.805862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fb77e3-b5ac-43ab-aab2-c19da8148839 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:38.813234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:38.819581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:38.819581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:38.819581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] PATH_INFO: `/v3/projects/a6012109e5714dd3b7beaa6e8ce5a1ae/users/08abaac60a804230a61f50422da0a903/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:38.835608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6012109e5714dd3b7beaa6e8ce5a1ae, user_id=08abaac60a804230a61f50422da0a903, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:38.838632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:38.996732 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1000/1988] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:38 2022] PUT /identity/v3/projects/a6012109e5714dd3b7beaa6e8ce5a1ae/users/08abaac60a804230a61f50422da0a903/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:39.002341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:39.016345 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf375e37-2fc2-4374-af0e-1db8940f0ec4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:39.116253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb825d0-e2b3-458b-8c10-a86b0280f6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:39.117453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb825d0-e2b3-458b-8c10-a86b0280f6f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:39.117863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb825d0-e2b3-458b-8c10-a86b0280f6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:39.118512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb825d0-e2b3-458b-8c10-a86b0280f6f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:39.118988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb825d0-e2b3-458b-8c10-a86b0280f6f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:39.121025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb825d0-e2b3-458b-8c10-a86b0280f6f8 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:39.136659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 989/1989] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:39.144514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:39.151885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be32717a-7967-4d84-b943-5a72ccf18a0f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:39.157170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:39.157825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:39.158100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:39.158369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] PATH_INFO: `/v3/projects/a6012109e5714dd3b7beaa6e8ce5a1ae/users/08abaac60a804230a61f50422da0a903/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:39.160276 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6012109e5714dd3b7beaa6e8ce5a1ae, user_id=08abaac60a804230a61f50422da0a903, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:39.164070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:39.183402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1001/1990] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:39 2022] PUT /identity/v3/projects/a6012109e5714dd3b7beaa6e8ce5a1ae/users/08abaac60a804230a61f50422da0a903/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:39.199267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:39.199805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:39.200416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:39.230241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] MFA Rules not processed for user `08abaac60a804230a61f50422da0a903`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:39.232077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:39.239591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:39.338292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 990/1991] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:39.344796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:39.346868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:39.450357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f052a-1018-47c4-be06-37d4210e7786 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:39.457623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:39.458681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:39.459596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:39.459867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:39.606167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:39.606167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:39.616833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1002/1992] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:39.762292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:39.764072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:39.776353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450c0a9-43eb-4568-b527-3189f3207746 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:39.781640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:39.782960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:39.782960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:39.783147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:39.785142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:39.861875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:39.863520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:39.874279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 991/1993] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:40.589804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:40.698315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:41.338552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:41.402009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:41.699715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:41.944317 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:42.303945 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:42.418177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:42.514880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=76,keystone:DELETE=22,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:42.838937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:42.841583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:42.848205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720023b-2c93-46aa-a4bc-8984bbb72e32 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:42.852286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:42.852981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:42.853324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:42.853666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:42.918507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:42.921281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:42.930412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1003/1994] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:42.994599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:43.595565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:43.909558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:43.978253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:44.961151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:45.084530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:45.093945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7fd48-aa1d-47bd-a9a8-915c4f2fcdbf service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:45.097760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3ba3-8970-436b-8ec9-4c62fc9db931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:45.098378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3ba3-8970-436b-8ec9-4c62fc9db931 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:45.098661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3ba3-8970-436b-8ec9-4c62fc9db931 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:45.098970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3ba3-8970-436b-8ec9-4c62fc9db931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:45.108570 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3ba3-8970-436b-8ec9-4c62fc9db931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:45.110007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3ba3-8970-436b-8ec9-4c62fc9db931 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:45.115718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:45.119528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 992/1995] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:45.355440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:45.543753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:45.784260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:45.981948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:46.378291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:46.528652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:46.884981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:46.943676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:47.437070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:47.438520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:47.518158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc450b-5fa2-4301-b432-3574c1b38652 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:47.522811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:47.523456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:47.523748 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:47.524022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] PATH_INFO: `/v3/users/f93839f7aec347b6a3004537dcfebf2d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:47.525159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93839f7aec347b6a3004537dcfebf2d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:47.527069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:47.694617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:47.829842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:47.845799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93839f7aec347b6a3004537dcfebf2d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:47.846519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93839f7aec347b6a3004537dcfebf2d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:47.847119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93839f7aec347b6a3004537dcfebf2d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:47.854476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93839f7aec347b6a3004537dcfebf2d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.000905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93839f7aec347b6a3004537dcfebf2d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:48.018714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1004/1996] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:47 2022] DELETE /identity/v3/users/f93839f7aec347b6a3004537dcfebf2d => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:48.023216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.023499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.023774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.042660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:48.044521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:48.104551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 993/1997] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:48.117618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:48.119060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:48.176742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ac658-d98c-49f3-853d-b767db23a5c3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:48.180385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:48.181060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.181256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.181524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.191126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:48.192581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:48.202734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1005/1998] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:48.839312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:48.842877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:48.856482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be3b8b5-a9c4-4bad-bc01-fd38c40b5451 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:48.861605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:48.862471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:48.862752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:48.863028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] PATH_INFO: `/v3/projects/e7ab78a6cf674ee8b4248a773c44dcf4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:48.864200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ab78a6cf674ee8b4248a773c44dcf4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:48.866245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:48.972499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:49.031403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ab78a6cf674ee8b4248a773c44dcf4', 'request_id': 'req-34f12612-33be-4297-840d-e5ba10176c26'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.039830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ab78a6cf674ee8b4248a773c44dcf4', 'request_id': 'req-34f12612-33be-4297-840d-e5ba10176c26'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.046870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ab78a6cf674ee8b4248a773c44dcf4', 'request_id': 'req-34f12612-33be-4297-840d-e5ba10176c26'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.149165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] The token cache is being invalidate because project e7ab78a6cf674ee8b4248a773c44dcf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:49.150122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.150579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.152249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 994/1999] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:48 2022] DELETE /identity/v3/projects/e7ab78a6cf674ee8b4248a773c44dcf4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:49.156422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:49.166198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ca41e-d363-4a3b-828e-e386c516c9a7 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:49.232383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:49.233284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:49.233561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:49.233859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] PATH_INFO: `/v3/users/4e3dfd0ce4b34291896edb1cdb48ceff` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:49.234987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3dfd0ce4b34291896edb1cdb48ceff)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:49.237023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:49.460792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3dfd0ce4b34291896edb1cdb48ceff'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.461512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3dfd0ce4b34291896edb1cdb48ceff'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.462256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3dfd0ce4b34291896edb1cdb48ceff'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.469902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3dfd0ce4b34291896edb1cdb48ceff'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.548715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-44560bf3-7847-4ff9-be0c-e5c679fab62c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3dfd0ce4b34291896edb1cdb48ceff'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:49.574603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1006/2000] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:49 2022] DELETE /identity/v3/users/4e3dfd0ce4b34291896edb1cdb48ceff => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:49.645102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:49.842350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:50.246274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.248581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:50.263165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f12612-33be-4297-840d-e5ba10176c26 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.334377 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.335097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.335422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.335751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] PATH_INFO: `/v3/projects/72fbe5f3e01443478011e69c4d845daf` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.336875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72fbe5f3e01443478011e69c4d845daf)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.339451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.407200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:50.461620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72fbe5f3e01443478011e69c4d845daf', 'request_id': 'req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.470218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72fbe5f3e01443478011e69c4d845daf', 'request_id': 'req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.476898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72fbe5f3e01443478011e69c4d845daf', 'request_id': 'req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.525094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:50.558736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] The token cache is being invalidate because project 72fbe5f3e01443478011e69c4d845daf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:50.559023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.559717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd4c2a84-0f06-4d89-99e5-d5a6448f35b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:50.561242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 995/2001] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:50 2022] DELETE /identity/v3/projects/72fbe5f3e01443478011e69c4d845daf => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:50.567329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.567638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.567923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.591430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:50.591430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:50.664839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1007/2002] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:50.679456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.679743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.680062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.701297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:50.705366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:50.711208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 996/2003] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:50.716296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.726292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda2c78-77c4-4c22-8779-675e7be47ed0 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.731558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.732230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.732510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.732780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.733422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:50.733733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.735307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.747227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1008/2004] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:50.752035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.761839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3070559-cfe4-4f8d-927e-6880c7b86221 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.767264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.767874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.768187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.768455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.769093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:50.771070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:50.882378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 997/2005] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:50 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:50.886391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:50.900163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f9fb1-94e2-4a2f-877d-a608330012a7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:50.995529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:50.996181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:50.996472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:50.997220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:50.999073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.001216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.138161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1009/2006] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:50 2022] POST /identity/v3/users => generated 285 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:51.142699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.153010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e95d4-0e8d-414b-a556-3f2c0efa0895 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.157878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.158518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.158803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.159075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.159717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.161337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.172780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 998/2007] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:51.180730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.192965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7377c9-d465-41ed-bfcf-7cdea4fa5b7c admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.198203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b2225-7eef-4428-ac02-b26dd329735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.199275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2225-7eef-4428-ac02-b26dd329735e admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.199710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2225-7eef-4428-ac02-b26dd329735e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.200088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b2225-7eef-4428-ac02-b26dd329735e admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.200931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b2225-7eef-4428-ac02-b26dd329735e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.202548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b2225-7eef-4428-ac02-b26dd329735e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.220888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1010/2008] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:51.225229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.237588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42901d-0185-40a9-8346-a8b6cf737df9 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.245655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.246456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.247720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.247720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] PATH_INFO: `/v3/projects/62a0053b8c614c36b30d71654e2b3ad0/users/20336c8417b24938a4b5aca538d72727/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.261406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:51.266382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a0053b8c614c36b30d71654e2b3ad0, user_id=20336c8417b24938a4b5aca538d72727, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.269373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.413303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 999/2009] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:51 2022] PUT /identity/v3/projects/62a0053b8c614c36b30d71654e2b3ad0/users/20336c8417b24938a4b5aca538d72727/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:51.420211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.420512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.420783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.473612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] MFA Rules not processed for user `20336c8417b24938a4b5aca538d72727`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:51.474685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:51.501072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:51.558157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1011/2010] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:51.571333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:51.573358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:51.657226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215ad30-11cd-4659-98ed-a4a4ef6b298b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:51.661461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:51.662128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:51.662411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:51.662683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:51.671916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:51.673530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:51.682298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1000/2011] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:53.733404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.734889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:53.808157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d3c9b-1e02-483c-91b2-218db0245e96 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.814327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.814980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.819300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.819300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] PATH_INFO: `/v3/users/2386a816917a4c3d97f394b07d901386` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.819300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2386a816917a4c3d97f394b07d901386)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.819300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:53.847081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:53.848636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:53.863549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e7470-b489-487f-9522-9a57db6d7853 service glance] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:53.868477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:53.869111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:53.869388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:53.869664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] PATH_INFO: `/v3/users/176eb10729624b83a330fa7cbd6e4d18` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:53.870882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176eb10729624b83a330fa7cbd6e4d18)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:53.872900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:54.173300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2386a816917a4c3d97f394b07d901386'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.173949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2386a816917a4c3d97f394b07d901386'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.175423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2386a816917a4c3d97f394b07d901386'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.182440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2386a816917a4c3d97f394b07d901386'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.213446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176eb10729624b83a330fa7cbd6e4d18'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.214093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176eb10729624b83a330fa7cbd6e4d18'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.214736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176eb10729624b83a330fa7cbd6e4d18'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.222282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176eb10729624b83a330fa7cbd6e4d18'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.262918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5dfe2065-a507-4d4d-a565-3c891daf26b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2386a816917a4c3d97f394b07d901386'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.282600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1012/2012] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:53 2022] DELETE /identity/v3/users/2386a816917a4c3d97f394b07d901386 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:54.332207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176eb10729624b83a330fa7cbd6e4d18'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.354965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1001/2013] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:53 2022] DELETE /identity/v3/users/176eb10729624b83a330fa7cbd6e4d18 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:54.359697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:54.359986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:54.360296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:54.384065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:54.385400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:54.421449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:54.451335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1013/2014] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:54.461115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:54.462566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:54.524798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da89ea25-c244-4615-9d05-622eb8614e24 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:54.529959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32709c36-93ef-433b-a357-9b9870c1d2c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:54.530569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709c36-93ef-433b-a357-9b9870c1d2c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:54.530848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709c36-93ef-433b-a357-9b9870c1d2c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:54.531119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32709c36-93ef-433b-a357-9b9870c1d2c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:54.537407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:54.541776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:54.549260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32709c36-93ef-433b-a357-9b9870c1d2c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:54.551867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32709c36-93ef-433b-a357-9b9870c1d2c6 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:54.557038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7c281-a2db-4cf1-886c-763d17d59442 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:54.566630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1002/2015] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:54.569339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:54.569995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:54.570269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:54.575829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] PATH_INFO: `/v3/projects/ac78c80460af43e0aa28dc607af188d5` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:54.575829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac78c80460af43e0aa28dc607af188d5)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:54.575829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:54.705864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac78c80460af43e0aa28dc607af188d5', 'request_id': 'req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.711473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac78c80460af43e0aa28dc607af188d5', 'request_id': 'req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.717856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac78c80460af43e0aa28dc607af188d5', 'request_id': 'req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.813946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] The token cache is being invalidate because project ac78c80460af43e0aa28dc607af188d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:54.813946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.813946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5e3d4955-8ef7-4a2f-8454-8bdf8d7919a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:54.815671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1014/2016] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:54 2022] DELETE /identity/v3/projects/ac78c80460af43e0aa28dc607af188d5 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:54.820646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:54.820932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:54.821207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:54.836424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:54.842745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:54.844109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:54.909082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1003/2017] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:54.916888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:54.917178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:54.917479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:54.950632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:54.952716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:54.962473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1015/2018] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:54.968112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:54.978548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f9a68-f533-474e-93db-ab96c1525ee7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:54.984297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:54.984920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:54.985219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:54.985489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:54.986308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:54.986622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:54.988370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.001219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1004/2019] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:23:55.005562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.015046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7998-0863-446a-bc79-60770ce88a9a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.019785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.020413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.020709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.020959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.021590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.023596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.075107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.076933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:55.092291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e50617-eb02-4f45-98e9-69b9551d4b7a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.097188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.097869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.098170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.105700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] PATH_INFO: `/v3/projects/5d18ca74104d4cbb93e2274ef088b2c3` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.105700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d18ca74104d4cbb93e2274ef088b2c3)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.105700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.133758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1016/2020] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:55 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:55.137724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.204308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c902e5-01c6-439e-ab7f-ffd4bc5eb846 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.219346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.219346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.219346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.219346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.219753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.219753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.257275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d18ca74104d4cbb93e2274ef088b2c3', 'request_id': 'req-1a786d83-6e1a-4afb-8a3d-b81bc797b020'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.263936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d18ca74104d4cbb93e2274ef088b2c3', 'request_id': 'req-1a786d83-6e1a-4afb-8a3d-b81bc797b020'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.270309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d18ca74104d4cbb93e2274ef088b2c3', 'request_id': 'req-1a786d83-6e1a-4afb-8a3d-b81bc797b020'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.345241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1017/2021] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:55 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:55.348116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.358509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d40974-939d-44c0-a83f-1e34b2cf00ec admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.378301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] The token cache is being invalidate because project 5d18ca74104d4cbb93e2274ef088b2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:55.385608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.386625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.389138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1005/2022] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:55 2022] DELETE /identity/v3/projects/5d18ca74104d4cbb93e2274ef088b2c3 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:55.391988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:55.394392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.395890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:55.426233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.426869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.427127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.427397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.428059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.429652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.439599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1018/2023] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:55.443519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.445214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:55.458701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fac1d1-309a-4805-973d-9130131497f3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.463858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.464516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.464795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.465065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.465255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a786d83-6e1a-4afb-8a3d-b81bc797b020 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.466264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.467852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.470657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.471288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.471616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.471896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] PATH_INFO: `/v3/users/cea04a2f4514429a8ef07ea0a03f15f9` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.473174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cea04a2f4514429a8ef07ea0a03f15f9)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.475610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.478749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1019/2024] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:55.483702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.493471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf4c520-4e02-4144-96c6-d20c591d4b51 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.498445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3072ba0-f150-4d62-bb87-1a25d8f891da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.499118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072ba0-f150-4d62-bb87-1a25d8f891da admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.499393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072ba0-f150-4d62-bb87-1a25d8f891da admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.499858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3072ba0-f150-4d62-bb87-1a25d8f891da admin admin] PATH_INFO: `/v3/projects/b572cbc25b254d6bb9c4c94e4e97608f/users/37219377c1764896ad0f5bc10cd862fc/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.499976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:55.514674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3072ba0-f150-4d62-bb87-1a25d8f891da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b572cbc25b254d6bb9c4c94e4e97608f, user_id=37219377c1764896ad0f5bc10cd862fc, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.517328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3072ba0-f150-4d62-bb87-1a25d8f891da admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.596460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1020/2025] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:55 2022] PUT /identity/v3/projects/b572cbc25b254d6bb9c4c94e4e97608f/users/37219377c1764896ad0f5bc10cd862fc/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:55.606967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.607536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.607815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.634547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] MFA Rules not processed for user `37219377c1764896ad0f5bc10cd862fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:55.635744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:55.676862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cea04a2f4514429a8ef07ea0a03f15f9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.677531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cea04a2f4514429a8ef07ea0a03f15f9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.678206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cea04a2f4514429a8ef07ea0a03f15f9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.685230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cea04a2f4514429a8ef07ea0a03f15f9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.701113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1021/2026] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:55.710949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:55.712315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:55.771679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cea04a2f4514429a8ef07ea0a03f15f9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:55.784317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafce765-8e52-4c3b-857b-4c48253295d7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:55.790030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:55.790585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1006/2027] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:55 2022] DELETE /identity/v3/users/cea04a2f4514429a8ef07ea0a03f15f9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:55.790705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:55.790898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:55.791166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:55.799975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:55.801639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:55.814625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1022/2028] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:56.049083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:56.049083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:56.122817 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57373-8bcb-488d-926f-2958190c7822 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:56.127437 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:56.128102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:56.128384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:56.128657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] PATH_INFO: `/v3/users/37219377c1764896ad0f5bc10cd862fc` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:56.129781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=37219377c1764896ad0f5bc10cd862fc)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:56.131906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:56.326850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:56.333355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:56.338412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:56.392326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37219377c1764896ad0f5bc10cd862fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.393011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37219377c1764896ad0f5bc10cd862fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.394134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37219377c1764896ad0f5bc10cd862fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.402443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37219377c1764896ad0f5bc10cd862fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.423728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1e729-f2ed-453f-be61-9bba0a5604e6 service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:56.428282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:56.428896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:56.429167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:56.429434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] PATH_INFO: `/v3/projects/19b08fde07e942ada3316e7c3e31af52` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:56.430533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b08fde07e942ada3316e7c3e31af52)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:56.432436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:56.547418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a7f1ea79-2f65-4303-83c4-ccdaabae9f9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37219377c1764896ad0f5bc10cd862fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.565283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1007/2029] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:56 2022] DELETE /identity/v3/users/37219377c1764896ad0f5bc10cd862fc => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:56.570586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:56.570865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:56.571190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:56.592463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:56.593672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:56.594767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b08fde07e942ada3316e7c3e31af52', 'request_id': 'req-036ab192-0c97-4677-82c8-ed7b715fa1aa'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.600636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b08fde07e942ada3316e7c3e31af52', 'request_id': 'req-036ab192-0c97-4677-82c8-ed7b715fa1aa'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.607890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b08fde07e942ada3316e7c3e31af52', 'request_id': 'req-036ab192-0c97-4677-82c8-ed7b715fa1aa'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.652531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:56.656033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1008/2030] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:56.668801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:56.670241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:56.690545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] The token cache is being invalidate because project 19b08fde07e942ada3316e7c3e31af52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:56.690835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.694829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:56.694829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1023/2031] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:56 2022] DELETE /identity/v3/projects/19b08fde07e942ada3316e7c3e31af52 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:56.697438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:56.698785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:56.712231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ab192-0c97-4677-82c8-ed7b715fa1aa admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:56.719566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:56.723481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:56.723481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:56.723481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] PATH_INFO: `/v3/users/08abaac60a804230a61f50422da0a903` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:56.723481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08abaac60a804230a61f50422da0a903)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:56.724198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:56.728032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617870e9-e3fc-43b3-b300-4e28941669fa None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:56.732485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:56.733142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:56.733417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:56.733684 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:56.735555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:56.747715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:56.749283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:56.758845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1009/2032] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:57.002934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08abaac60a804230a61f50422da0a903'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.005954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08abaac60a804230a61f50422da0a903'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.005954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08abaac60a804230a61f50422da0a903'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.011500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08abaac60a804230a61f50422da0a903'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.115036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1d55d689-81d2-41d1-94cb-952c66463412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08abaac60a804230a61f50422da0a903'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.136355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1024/2033] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:56 2022] DELETE /identity/v3/users/08abaac60a804230a61f50422da0a903 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:57.294223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:57.295649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:57.318938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb67db-fd5a-4143-b8b6-0a6da1afb9ed service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:57.378522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:57.379160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:57.379440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:57.379711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] PATH_INFO: `/v3/projects/b572cbc25b254d6bb9c4c94e4e97608f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:57.380876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b572cbc25b254d6bb9c4c94e4e97608f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:57.382767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:57.508234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b572cbc25b254d6bb9c4c94e4e97608f', 'request_id': 'req-5b57509c-3c90-4125-bc87-786b610aeca1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.515006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b572cbc25b254d6bb9c4c94e4e97608f', 'request_id': 'req-5b57509c-3c90-4125-bc87-786b610aeca1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.521609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b572cbc25b254d6bb9c4c94e4e97608f', 'request_id': 'req-5b57509c-3c90-4125-bc87-786b610aeca1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.566555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:57.622668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] The token cache is being invalidate because project b572cbc25b254d6bb9c4c94e4e97608f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:57.623022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.623731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.625266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1010/2034] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:57 2022] DELETE /identity/v3/projects/b572cbc25b254d6bb9c4c94e4e97608f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:57.630262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420d413-0e32-4060-84be-fe861323887d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:57.630548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420d413-0e32-4060-84be-fe861323887d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:57.630828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a420d413-0e32-4060-84be-fe861323887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:57.639689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:57.652360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a420d413-0e32-4060-84be-fe861323887d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:57.654150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a420d413-0e32-4060-84be-fe861323887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:57.724332 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:57.725886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:57.741348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1025/2035] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:57.749453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:57.749741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:57.750016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:57.772346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:57.773568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:57.782913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1026/2036] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:57.786815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:57.796452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153cce1-5874-45bd-b6e6-1ad6c268beee None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:57.801368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:57.801967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:57.802244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:57.802516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:57.803160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:57.803463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:57.803846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57509c-3c90-4125-bc87-786b610aeca1 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:57.805299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:57.808647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:57.809322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:57.811629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:57.811629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] PATH_INFO: `/v3/projects/a6012109e5714dd3b7beaa6e8ce5a1ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:57.811629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6012109e5714dd3b7beaa6e8ce5a1ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:57.812915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:57.817597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1027/2037] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:23:57.821372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:57.832768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34147bd2-de7b-4367-a6bf-dd1bdd7235ef admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:57.838986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:57.840143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:57.841006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:57.841509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:57.842237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:57.844235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:57.972359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6012109e5714dd3b7beaa6e8ce5a1ae', 'request_id': 'req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.978516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6012109e5714dd3b7beaa6e8ce5a1ae', 'request_id': 'req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:57.984925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6012109e5714dd3b7beaa6e8ce5a1ae', 'request_id': 'req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:58.014764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1028/2038] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:57 2022] POST /identity/v3/projects => generated 403 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:58.016752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.077964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4647-3782-4e52-803c-7322478a4992 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.082586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.083181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.083456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.083723 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.084359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.086242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.097945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] The token cache is being invalidate because project a6012109e5714dd3b7beaa6e8ce5a1ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:23:58.098299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:58.099013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-59cc9d9f-5ba3-45f4-97b8-50b3f4488b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:23:58.100562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1011/2039] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:23:57 2022] DELETE /identity/v3/projects/a6012109e5714dd3b7beaa6e8ce5a1ae => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:58.105188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a7ab9-ceb6-4ec5-b8b7-382f32fbd911 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.105467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a7ab9-ceb6-4ec5-b8b7-382f32fbd911 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.105735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a7ab9-ceb6-4ec5-b8b7-382f32fbd911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.126189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c93a7ab9-ceb6-4ec5-b8b7-382f32fbd911 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:58.127592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c93a7ab9-ceb6-4ec5-b8b7-382f32fbd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:58.136309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1012/2040] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:58.151401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.151717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.152013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.176131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:58.177347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:58.185475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1013/2041] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:58.192495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.203591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c321f46-ca46-456a-92fc-113fb7a8d0f2 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.208267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.208908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.209164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.209432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.210068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:23:58.210364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.211946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.217247 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1029/2042] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/users => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:58.224273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1014/2043] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:23:58.226066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.227738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:58.228237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.238090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a30e-f5ce-480e-b71f-60b0169ef49d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.240685 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599fa0f8-7da3-474b-9463-346adab19b90 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.242768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.243349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.243626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.243897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.244590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.245633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.246231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.246467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.246564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.246791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.247523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.249127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.259306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1030/2044] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:58.264521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.275060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c6f93a-3744-404a-aa8c-ca9ce6daa2f1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.280384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.280991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.281266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.281537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.282170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.283759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.295752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1031/2045] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:58.300015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.310739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3ae81-d553-4e3c-b3c0-9de8a6b887d3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.315759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373162ea-b854-4f3c-a0de-c23c830909db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.316555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373162ea-b854-4f3c-a0de-c23c830909db admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.317069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373162ea-b854-4f3c-a0de-c23c830909db admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.317599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373162ea-b854-4f3c-a0de-c23c830909db admin admin] PATH_INFO: `/v3/projects/10bc1c79d17a46dd9fab72de360241b4/users/6c94aa8096304f9e9231104d904b02e8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.342950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373162ea-b854-4f3c-a0de-c23c830909db admin admin] RBAC: Authorizing `identity:create_grant(project_id=10bc1c79d17a46dd9fab72de360241b4, user_id=6c94aa8096304f9e9231104d904b02e8, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.345592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373162ea-b854-4f3c-a0de-c23c830909db admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.350657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1015/2046] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:23:58.354849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.415567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff6916-4119-4aa0-a0cf-2713760dd9b6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.420456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.421059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.421336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.421605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.422368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.424310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.430614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1032/2047] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:58 2022] PUT /identity/v3/projects/10bc1c79d17a46dd9fab72de360241b4/users/6c94aa8096304f9e9231104d904b02e8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:58.436079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:58.447351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.447637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.447907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.476202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] MFA Rules not processed for user `6c94aa8096304f9e9231104d904b02e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:58.478086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:58.540045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1033/2048] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:58.544455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.553648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1016/2049] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:58.558884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.569833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9668b-5398-4bd3-a22a-527fc8f9c486 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.609618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22a9d9-c7f9-4a84-b2c5-073b99009e55 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.614331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.615031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.615350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.615660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.616547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.618678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.626900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.627537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.627811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.628114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.628760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.630390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.641277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1017/2050] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:58.645293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.655188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156ed64-bcd1-4e55-92b4-edd351fe8f15 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.659763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.660365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.660642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.660908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.661526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.663087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.673341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1018/2051] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:58.678321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.689338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e80f7f-ee3f-4628-a93f-c8ccf6b0a848 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.694349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3642d20d-ca9c-4a47-97f1-cf9a2e4e851c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.697860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642d20d-ca9c-4a47-97f1-cf9a2e4e851c admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.697860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642d20d-ca9c-4a47-97f1-cf9a2e4e851c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.697860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3642d20d-ca9c-4a47-97f1-cf9a2e4e851c admin admin] PATH_INFO: `/v3/projects/29447f0b81cb4a8bbeab875115c2b542/users/a1c55eb65703476994509add9731e0db/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.711445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642d20d-ca9c-4a47-97f1-cf9a2e4e851c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29447f0b81cb4a8bbeab875115c2b542, user_id=a1c55eb65703476994509add9731e0db, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.714117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3642d20d-ca9c-4a47-97f1-cf9a2e4e851c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.727671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1034/2052] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/projects => generated 405 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:58.732414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.798602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b734496-20d0-4fe6-890e-e962c968469a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:58.803601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:58.804327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.804687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.804957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.805659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:58.807494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:58.845702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1019/2053] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:58 2022] PUT /identity/v3/projects/29447f0b81cb4a8bbeab875115c2b542/users/a1c55eb65703476994509add9731e0db/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:58.856788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:58.857364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:58.857459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:58.888098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] MFA Rules not processed for user `a1c55eb65703476994509add9731e0db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:58.888098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:58.953691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1020/2054] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:23:58.955197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1035/2055] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:23:58 2022] POST /identity/v3/users => generated 309 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:23:58.959070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.964882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:58.965001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:58.966983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:59.024419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21897d2a-4ed1-48eb-bb21-4f702087443c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.025838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f1fb08-d519-4db1-93c9-b7bf84b03fba admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.029487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.030086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.030364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.030631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.031264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.032880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.042492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1021/2056] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:23:59.047953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:59.058180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d3c5d-c525-456c-ad32-5a9b931c3720 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.066884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.067565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.067853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.068166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] PATH_INFO: `/v3/projects/436bf21021fc48bc86808e44a9d942f1/users/3eed43eab39d4260b5e35b89657b647f/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.080878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.083235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.083235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.083235 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.083472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] RBAC: Authorizing `identity:create_grant(project_id=436bf21021fc48bc86808e44a9d942f1, user_id=3eed43eab39d4260b5e35b89657b647f, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.086114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.090791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.092326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.100907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1036/2057] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:23:59.218486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1022/2058] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:59 2022] PUT /identity/v3/projects/436bf21021fc48bc86808e44a9d942f1/users/3eed43eab39d4260b5e35b89657b647f/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:23:59.229167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:59.242205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf82d6-02ff-4759-9787-25344544e300 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.299677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679d9e3-9a0a-49e8-add7-dcdb872e7cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.300312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d9e3-9a0a-49e8-add7-dcdb872e7cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.300585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d9e3-9a0a-49e8-add7-dcdb872e7cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.300852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679d9e3-9a0a-49e8-add7-dcdb872e7cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.301523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679d9e3-9a0a-49e8-add7-dcdb872e7cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.307095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679d9e3-9a0a-49e8-add7-dcdb872e7cfa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.318145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1037/2059] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:23:59.322428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:59.334274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241c0be-4e78-4220-afa3-cbe2e45543da admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.339182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.339832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.340244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.340634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] PATH_INFO: `/v3/projects/436bf21021fc48bc86808e44a9d942f1/users/3eed43eab39d4260b5e35b89657b647f/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.342689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=436bf21021fc48bc86808e44a9d942f1, user_id=3eed43eab39d4260b5e35b89657b647f, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.345495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.355639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1023/2060] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:23:59 2022] PUT /identity/v3/projects/436bf21021fc48bc86808e44a9d942f1/users/3eed43eab39d4260b5e35b89657b647f/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:23:59.366199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.366476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.366744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.398322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] MFA Rules not processed for user `3eed43eab39d4260b5e35b89657b647f`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:23:59.399572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:59.469029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1038/2061] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:23:59 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:23:59.479756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:59.482159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:23:59.538088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:59.543078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e4da9-8b59-4ac4-99c5-c1705047c5bb admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.547365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62a401-c22f-4164-ba94-c15a866a6512 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.548111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62a401-c22f-4164-ba94-c15a866a6512 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.548408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62a401-c22f-4164-ba94-c15a866a6512 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.548677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62a401-c22f-4164-ba94-c15a866a6512 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.557671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62a401-c22f-4164-ba94-c15a866a6512 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.559711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62a401-c22f-4164-ba94-c15a866a6512 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:23:59.569034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1024/2062] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:23:59 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:23:59.658057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:23:59.725218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:23:59.792470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49466590-1e5e-48a7-bb9f-97dbddc96ed7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:23:59.867329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:23:59.867329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:23:59.867329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:23:59.867329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] PATH_INFO: `/v3/users/6c94aa8096304f9e9231104d904b02e8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:23:59.867962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c94aa8096304f9e9231104d904b02e8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:23:59.867962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:00.288448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c94aa8096304f9e9231104d904b02e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.289033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c94aa8096304f9e9231104d904b02e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.289507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c94aa8096304f9e9231104d904b02e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.297040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c94aa8096304f9e9231104d904b02e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.389517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c94aa8096304f9e9231104d904b02e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:00.407090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1039/2063] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:23:59 2022] DELETE /identity/v3/users/6c94aa8096304f9e9231104d904b02e8 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:00.411665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:00.412174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:00.412275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:00.431314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:00.432612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:00.492344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1025/2064] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:00.503295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:00.504703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:00.549854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:00.562617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9315e25a-8dbb-44b9-a1d1-c61feffc5bed admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:00.566341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:00.566937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:00.567213 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:00.567483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:00.577111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:00.578569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:00.587727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1040/2065] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:01.059630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.061143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:01.074192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c7d2e-2576-4f72-9373-c1818467df21 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:01.080351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:01.081080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.081360 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.081633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] PATH_INFO: `/v3/projects/10bc1c79d17a46dd9fab72de360241b4` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.083980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10bc1c79d17a46dd9fab72de360241b4)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:01.086004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:01.095217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:01.187690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10bc1c79d17a46dd9fab72de360241b4', 'request_id': 'req-55fa5827-1d84-4762-996c-28e7dc938129'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.194535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10bc1c79d17a46dd9fab72de360241b4', 'request_id': 'req-55fa5827-1d84-4762-996c-28e7dc938129'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.203395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10bc1c79d17a46dd9fab72de360241b4', 'request_id': 'req-55fa5827-1d84-4762-996c-28e7dc938129'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.298221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] The token cache is being invalidate because project 10bc1c79d17a46dd9fab72de360241b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:01.298525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.300027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55fa5827-1d84-4762-996c-28e7dc938129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.301027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1026/2066] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:01 2022] DELETE /identity/v3/projects/10bc1c79d17a46dd9fab72de360241b4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:01.305022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:01.315636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17085b25-583e-4eb3-8f42-82cd887f0a15 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:01.323630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:01.375913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:01.388051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:01.388051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:01.388051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] PATH_INFO: `/v3/users/3eed43eab39d4260b5e35b89657b647f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:01.388051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eed43eab39d4260b5e35b89657b647f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:01.388051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:01.665320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eed43eab39d4260b5e35b89657b647f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.666088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eed43eab39d4260b5e35b89657b647f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.666693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eed43eab39d4260b5e35b89657b647f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.674069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eed43eab39d4260b5e35b89657b647f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.766372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eed43eab39d4260b5e35b89657b647f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:01.785853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1041/2067] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:01 2022] DELETE /identity/v3/users/3eed43eab39d4260b5e35b89657b647f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:02.060335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0f19d-4096-4646-a96e-b3d6eeec9662 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.060812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0f19d-4096-4646-a96e-b3d6eeec9662 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.060909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0f19d-4096-4646-a96e-b3d6eeec9662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.081088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-16b0f19d-4096-4646-a96e-b3d6eeec9662 None None] MFA Rules not processed for user `a1c55eb65703476994509add9731e0db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:02.082594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-16b0f19d-4096-4646-a96e-b3d6eeec9662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:02.143973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1027/2068] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:02.276068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.277542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:02.347483 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea15aa-6066-4e81-8109-a08878eabedb admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.352225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.352895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.353172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.353444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] PATH_INFO: `/v3/projects/436bf21021fc48bc86808e44a9d942f1` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.354572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] RBAC: Authorizing `identity:delete_project(project_id=436bf21021fc48bc86808e44a9d942f1)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.356879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.446567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:02.465561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '436bf21021fc48bc86808e44a9d942f1', 'request_id': 'req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:02.471731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '436bf21021fc48bc86808e44a9d942f1', 'request_id': 'req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:02.478260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '436bf21021fc48bc86808e44a9d942f1', 'request_id': 'req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:02.560698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] The token cache is being invalidate because project 436bf21021fc48bc86808e44a9d942f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:02.561091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:02.561722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9c51f4ef-4618-42e1-91dc-2e1fe4bfe94b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:02.563433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1042/2069] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:02 2022] DELETE /identity/v3/projects/436bf21021fc48bc86808e44a9d942f1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:02.568324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.574218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.574218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.597537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:02.598752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:02.667933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1028/2070] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:02.678467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da1147-669d-463c-b707-00a3a4568065 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.695350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da1147-669d-463c-b707-00a3a4568065 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.695350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da1147-669d-463c-b707-00a3a4568065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.700063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-68da1147-669d-463c-b707-00a3a4568065 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:02.701256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-68da1147-669d-463c-b707-00a3a4568065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:02.709370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1043/2071] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:02.713740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.723642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f374b-deb8-43da-97b6-7aa72ba4c9b6 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.728357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.728946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.729224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.729536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.730186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:02.730485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.732684 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.744287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1029/2072] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:24:02.749883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da1147-669d-463c-b707-00a3a4568065 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.759193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da1147-669d-463c-b707-00a3a4568065 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.764918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.765773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.766275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.766569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.767595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.769579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:02.874760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1044/2073] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:02.875849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:02.886849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c290aa-3707-4274-a706-42b9b1af65e6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:02.925889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:02.957898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:02.958516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:02.958790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:02.959195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:02.959935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:02.962157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.063644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1030/2074] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:02 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:24:03.067677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.078118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e180f-17b9-4386-9b69-36672b5e3eb1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.082642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.083237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.083513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.083782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.084454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.086182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.097285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1045/2075] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:03.101343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.112364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcc2c4-5d5f-407a-86cb-32159c924d8d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.117031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b0b02-ca4d-440c-9357-24084b6612f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.117684 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0b02-ca4d-440c-9357-24084b6612f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.117969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0b02-ca4d-440c-9357-24084b6612f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.118241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b0b02-ca4d-440c-9357-24084b6612f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.118888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b0b02-ca4d-440c-9357-24084b6612f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.120553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b0b02-ca4d-440c-9357-24084b6612f2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.130683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1031/2076] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:03.135044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.145258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5665fe-38e0-4392-818e-940594f20cfa admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.150328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.150991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.151251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.151517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] PATH_INFO: `/v3/projects/4863c62271644a67886b090356817bfd/users/2e1383c14f8a409385c85537732f69b8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.166899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4863c62271644a67886b090356817bfd, user_id=2e1383c14f8a409385c85537732f69b8, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.170659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.257592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1046/2077] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:03 2022] PUT /identity/v3/projects/4863c62271644a67886b090356817bfd/users/2e1383c14f8a409385c85537732f69b8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:03.268474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.268756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.269026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.297424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] MFA Rules not processed for user `2e1383c14f8a409385c85537732f69b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:03.298747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:03.363566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1032/2078] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:03.368407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.439641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c1a24-5b8e-4175-9364-3f224d2d70a1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.444442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.445128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.445407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.445665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.446295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.448268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.568103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1047/2079] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:03 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:03.572105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.584328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3580981-df87-4946-8ca9-d7ad197c7e3c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.643983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.645621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.645621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.645621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.645897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.647698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.751042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1033/2080] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:03 2022] POST /identity/v3/users => generated 300 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:24:03.755143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.768343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6debe2c-27e3-4c8f-83f8-e611b0277466 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.775524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.776165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.776448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.776719 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.777367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.778974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.791137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1048/2081] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:03.795537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.806144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4be180-4ae5-461d-b5f7-68cdb1412d72 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:03.813056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:03.814073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:03.814532 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:03.815199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] PATH_INFO: `/v3/projects/2f66c85e31544241b81a50813fbfcad3/users/bbe4a60015ae4032bac5d59ae837099b/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:03.832493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f66c85e31544241b81a50813fbfcad3, user_id=bbe4a60015ae4032bac5d59ae837099b, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:03.835118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:03.977192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1034/2082] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:03 2022] PUT /identity/v3/projects/2f66c85e31544241b81a50813fbfcad3/users/bbe4a60015ae4032bac5d59ae837099b/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:03.981414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:03.991613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa729a82-6b09-4047-9769-33c7b64924db admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.022141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:04.049785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f626fc-0a7c-4e8f-a644-146761593621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.050401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f626fc-0a7c-4e8f-a644-146761593621 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.050681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f626fc-0a7c-4e8f-a644-146761593621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.051099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f626fc-0a7c-4e8f-a644-146761593621 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.051771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f626fc-0a7c-4e8f-a644-146761593621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.053438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f626fc-0a7c-4e8f-a644-146761593621 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.067650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1049/2083] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:04.068878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.075805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:04.081779 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c57c7b-12e9-446d-93a9-432b5a6f5b11 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.088285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.089697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.089697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.090008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] PATH_INFO: `/v3/projects/2f66c85e31544241b81a50813fbfcad3/users/bbe4a60015ae4032bac5d59ae837099b/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.092764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f66c85e31544241b81a50813fbfcad3, user_id=bbe4a60015ae4032bac5d59ae837099b, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.095389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.105678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1035/2084] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:04 2022] PUT /identity/v3/projects/2f66c85e31544241b81a50813fbfcad3/users/bbe4a60015ae4032bac5d59ae837099b/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:04.118260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.118546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.118825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.148866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] MFA Rules not processed for user `bbe4a60015ae4032bac5d59ae837099b`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:04.150105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:04.219233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1050/2085] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:04.230354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.233190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:04.292584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf05c35-4412-409e-9700-3d11ad716e1a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.297209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c9fb7-f6c0-4b0f-b90b-328f521debe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.297812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c9fb7-f6c0-4b0f-b90b-328f521debe0 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.298087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c9fb7-f6c0-4b0f-b90b-328f521debe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.298418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c9fb7-f6c0-4b0f-b90b-328f521debe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.307320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c9fb7-f6c0-4b0f-b90b-328f521debe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.308777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c9fb7-f6c0-4b0f-b90b-328f521debe0 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.319225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1036/2086] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:04.349535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.414334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be07b8-d35b-4df0-bbe0-5fae68437071 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.424586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.425244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.425525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.425797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] PATH_INFO: `/v3/users/2e1383c14f8a409385c85537732f69b8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.426964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1383c14f8a409385c85537732f69b8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.429000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:04.642976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1383c14f8a409385c85537732f69b8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.643812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1383c14f8a409385c85537732f69b8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.644580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1383c14f8a409385c85537732f69b8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.651043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:04.653785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1383c14f8a409385c85537732f69b8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.770214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1383c14f8a409385c85537732f69b8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:04.798998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1051/2087] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:04 2022] DELETE /identity/v3/users/2e1383c14f8a409385c85537732f69b8 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:04.800619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.800904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.801173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.822239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:04.823476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:04.889216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1037/2088] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:04.907549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:04.908966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:04.978974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85673732-fec5-4784-a29e-a050715fdb96 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:04.982585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:04.983194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:04.983471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:04.983747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:04.992682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:04.994581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:05.005846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1052/2089] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:05.516894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.518477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:05.534009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3d85f7-8e11-413d-99a9-9e686f228597 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.541289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.575410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.575410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.575410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] PATH_INFO: `/v3/projects/4863c62271644a67886b090356817bfd` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.575410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4863c62271644a67886b090356817bfd)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.575410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:05.751353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4863c62271644a67886b090356817bfd', 'request_id': 'req-ccb9254b-11d2-4640-b1c2-388f60671ecc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.761423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4863c62271644a67886b090356817bfd', 'request_id': 'req-ccb9254b-11d2-4640-b1c2-388f60671ecc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.767925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4863c62271644a67886b090356817bfd', 'request_id': 'req-ccb9254b-11d2-4640-b1c2-388f60671ecc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.862110 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] The token cache is being invalidate because project 4863c62271644a67886b090356817bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:05.862562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.863296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:05.864891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1038/2090] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:05 2022] DELETE /identity/v3/projects/4863c62271644a67886b090356817bfd => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:05.868953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:05.878633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90bb43-7d03-401b-955e-5843ca133385 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:05.940494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:05.941130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:05.941409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:05.941688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] PATH_INFO: `/v3/users/bbe4a60015ae4032bac5d59ae837099b` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:05.942872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe4a60015ae4032bac5d59ae837099b)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:05.944889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:06.222644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe4a60015ae4032bac5d59ae837099b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:06.224351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe4a60015ae4032bac5d59ae837099b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:06.224948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe4a60015ae4032bac5d59ae837099b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:06.232609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe4a60015ae4032bac5d59ae837099b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:06.354516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-007d643c-a757-4cfc-b0d1-895c9ea737b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe4a60015ae4032bac5d59ae837099b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:06.372688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1053/2091] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:05 2022] DELETE /identity/v3/users/bbe4a60015ae4032bac5d59ae837099b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:06.373221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:06.973624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:06.975151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:06.990031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb9254b-11d2-4640-b1c2-388f60671ecc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:07.005558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:07.056353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:07.057051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.057330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.057601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] PATH_INFO: `/v3/projects/2f66c85e31544241b81a50813fbfcad3` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.058810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f66c85e31544241b81a50813fbfcad3)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:07.061718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:07.188118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f66c85e31544241b81a50813fbfcad3', 'request_id': 'req-0213e38a-fd8b-492b-9772-e36fe57a680e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:07.200132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f66c85e31544241b81a50813fbfcad3', 'request_id': 'req-0213e38a-fd8b-492b-9772-e36fe57a680e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:07.209454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f66c85e31544241b81a50813fbfcad3', 'request_id': 'req-0213e38a-fd8b-492b-9772-e36fe57a680e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:07.319171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] The token cache is being invalidate because project 2f66c85e31544241b81a50813fbfcad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:07.319171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:07.319171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0213e38a-fd8b-492b-9772-e36fe57a680e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:07.323967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1039/2092] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:06 2022] DELETE /identity/v3/projects/2f66c85e31544241b81a50813fbfcad3 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:07.330464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.330960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.331073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.377397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:07.379299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:07.448648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1054/2093] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:07.458006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.458354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.462282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.479041 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:07.480276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:07.489566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1040/2094] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:07.493544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:07.504099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5bf3cc-1240-4096-bbe2-3e60e80d5aae None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:07.508781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:07.518225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.518225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.518225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.518225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:07.518225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:07.518225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:07.528834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1055/2095] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:24:07.530119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:07.539430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96e1af-2de4-49f5-bcb1-a1d434713a5f None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:07.545078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:07.545687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.545994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.546269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.546908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:07.548902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:07.653832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1041/2096] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:07 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:07.658188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:07.670651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844cdc-ff16-4587-9b1f-110c7e305eb7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:07.767519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:07.768195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.768477 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.768752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.769393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:07.771317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:07.951144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1056/2097] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:07 2022] POST /identity/v3/users => generated 295 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:07.958248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:07.967550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e78903-861a-43c1-b861-43d8a2575b1d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:07.972275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:07.972909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:07.973190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:07.973456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:07.974101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:07.975715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:07.988194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1042/2098] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:07.995589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.006251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0efe20f-d5a1-4cfc-a7b5-a1a3bbbc43ae admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:08.011665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d0d5c-56d9-48bf-93cd-42b665cf0a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:08.013106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0d5c-56d9-48bf-93cd-42b665cf0a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.013697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0d5c-56d9-48bf-93cd-42b665cf0a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.014162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d0d5c-56d9-48bf-93cd-42b665cf0a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.014843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d0d5c-56d9-48bf-93cd-42b665cf0a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:08.017152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d0d5c-56d9-48bf-93cd-42b665cf0a59 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:08.033353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1057/2099] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:08.034626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.047193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853efcaa-8fa9-420a-a8b1-ba133b03c074 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:08.051933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:08.053298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.053582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.053857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] PATH_INFO: `/v3/projects/66edfbe9438a4fd0b86878f7dde4ba17/users/2a643f1e58474990b7a70b7cdb3c3996/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.072295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66edfbe9438a4fd0b86878f7dde4ba17, user_id=2a643f1e58474990b7a70b7cdb3c3996, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:08.075947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:08.187584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1043/2100] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:08 2022] PUT /identity/v3/projects/66edfbe9438a4fd0b86878f7dde4ba17/users/2a643f1e58474990b7a70b7cdb3c3996/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:08.194921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.195203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.195475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.228951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] MFA Rules not processed for user `2a643f1e58474990b7a70b7cdb3c3996`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:08.231324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:08.280283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:08.297657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1058/2101] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:08.302319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.350282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:08.376630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f6f2e-ff83-4b92-bfbd-2db0731d2467 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:08.385713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:08.390126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.390126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.390126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.390126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:08.391082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:08.505003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1044/2102] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:08 2022] POST /identity/v3/projects => generated 389 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:08.509096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.520907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a94681-8c05-4957-a52c-dcd34b258088 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:08.585661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:08.586306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.586579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.586850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.587542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:08.589460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:08.729090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1059/2103] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:08 2022] POST /identity/v3/users => generated 301 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:08.732462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.742957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6139037-44dd-48eb-83d4-ce4e50314655 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:08.748804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:08.749837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.750170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.750443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.751148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:08.752778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:08.762955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1045/2104] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:24:08.767539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.779700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ba28e-86ac-4a6e-b8c9-5be9613dfbeb admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:08.784900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:08.785706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:08.786111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:08.786529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] PATH_INFO: `/v3/projects/04f189e0d8ea4c3bbb8a5eaea942f3b7/users/4e4d62eab8274dfb94a7805c1a2b422d/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:08.804624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f189e0d8ea4c3bbb8a5eaea942f3b7, user_id=4e4d62eab8274dfb94a7805c1a2b422d, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:08.807288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:08.943442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1060/2105] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:08 2022] PUT /identity/v3/projects/04f189e0d8ea4c3bbb8a5eaea942f3b7/users/4e4d62eab8274dfb94a7805c1a2b422d/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:08.947583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:08.960129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1c7bf-907d-4b88-a9dc-cb7439a72371 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:09.018581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de08d85-e8f4-456c-9e6c-5ccb277377cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:09.019280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08d85-e8f4-456c-9e6c-5ccb277377cd admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.019371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08d85-e8f4-456c-9e6c-5ccb277377cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.019633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de08d85-e8f4-456c-9e6c-5ccb277377cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.020333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de08d85-e8f4-456c-9e6c-5ccb277377cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:09.025180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de08d85-e8f4-456c-9e6c-5ccb277377cd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:09.033233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1046/2106] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:09.037471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:09.048291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe16f4d-ab4c-4b32-8d3b-9024a752ac45 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:09.053019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:09.053653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.053924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.054188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] PATH_INFO: `/v3/projects/04f189e0d8ea4c3bbb8a5eaea942f3b7/users/4e4d62eab8274dfb94a7805c1a2b422d/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.056195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f189e0d8ea4c3bbb8a5eaea942f3b7, user_id=4e4d62eab8274dfb94a7805c1a2b422d, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:09.058910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:09.069603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1061/2107] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:09 2022] PUT /identity/v3/projects/04f189e0d8ea4c3bbb8a5eaea942f3b7/users/4e4d62eab8274dfb94a7805c1a2b422d/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:09.085801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.086083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.086384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.114791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] MFA Rules not processed for user `4e4d62eab8274dfb94a7805c1a2b422d`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:09.116259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:09.183085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1047/2108] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:09 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:09.201674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:09.203411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:09.261245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae16c78-7490-47f2-89e0-a8a14eeb5ab8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:09.265665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80fa55-484b-40ed-b47b-afe0f14f5b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:09.266272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80fa55-484b-40ed-b47b-afe0f14f5b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.266553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80fa55-484b-40ed-b47b-afe0f14f5b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.266820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80fa55-484b-40ed-b47b-afe0f14f5b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.275782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80fa55-484b-40ed-b47b-afe0f14f5b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:09.277170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80fa55-484b-40ed-b47b-afe0f14f5b7b service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:09.286991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1062/2109] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:09.424593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:09.473423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:09.494454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8817e6-f64e-431b-918c-91e77e7e870a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:09.499053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:09.499709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.499990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.500305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] PATH_INFO: `/v3/users/2a643f1e58474990b7a70b7cdb3c3996` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.501444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a643f1e58474990b7a70b7cdb3c3996)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:09.503487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:09.684473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a643f1e58474990b7a70b7cdb3c3996'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:09.685519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a643f1e58474990b7a70b7cdb3c3996'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:09.686229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a643f1e58474990b7a70b7cdb3c3996'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:09.693764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a643f1e58474990b7a70b7cdb3c3996'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:09.777768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a643f1e58474990b7a70b7cdb3c3996'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:09.800450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1048/2110] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:09 2022] DELETE /identity/v3/users/2a643f1e58474990b7a70b7cdb3c3996 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:09.803767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.804335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.804727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.826601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:09.827931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:09.870389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:09.889561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1063/2111] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:09.899334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:09.900753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:09.959284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5435971-ce7c-4b60-8e19-ece3c76904af admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:09.962885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:09.963481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:09.963757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:09.964075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:09.973917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:09.975352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:09.985037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1049/2112] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:10.413142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:10.414569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:10.427418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebf68ff-7155-4f93-9870-be40f36ee669 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:10.432078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:10.432830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:10.433069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:10.433341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] PATH_INFO: `/v3/projects/66edfbe9438a4fd0b86878f7dde4ba17` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:10.434490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66edfbe9438a4fd0b86878f7dde4ba17)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:10.436505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:10.544138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66edfbe9438a4fd0b86878f7dde4ba17', 'request_id': 'req-387b2d9e-4e83-4e42-9e09-c3c3cd181849'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.549560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66edfbe9438a4fd0b86878f7dde4ba17', 'request_id': 'req-387b2d9e-4e83-4e42-9e09-c3c3cd181849'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.555620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66edfbe9438a4fd0b86878f7dde4ba17', 'request_id': 'req-387b2d9e-4e83-4e42-9e09-c3c3cd181849'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.633523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] The token cache is being invalidate because project 66edfbe9438a4fd0b86878f7dde4ba17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:10.633737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.634379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.635915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1064/2113] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:10 2022] DELETE /identity/v3/projects/66edfbe9438a4fd0b86878f7dde4ba17 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:10.639969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:10.650269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c00e2-bd22-4415-ba04-72a0527d441d service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:10.708049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:10.708683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:10.708963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:10.709240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] PATH_INFO: `/v3/users/4e4d62eab8274dfb94a7805c1a2b422d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:10.710368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4d62eab8274dfb94a7805c1a2b422d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:10.712310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:10.713807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:10.897085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4d62eab8274dfb94a7805c1a2b422d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.897797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4d62eab8274dfb94a7805c1a2b422d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.898411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4d62eab8274dfb94a7805c1a2b422d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.911518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4d62eab8274dfb94a7805c1a2b422d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:10.988933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-92846ce1-c0f1-4d94-9c39-0556250eae6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4d62eab8274dfb94a7805c1a2b422d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:11.007688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1050/2114] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:10 2022] DELETE /identity/v3/users/4e4d62eab8274dfb94a7805c1a2b422d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:11.473085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:11.474684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:11.490819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b2d9e-4e83-4e42-9e09-c3c3cd181849 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:11.549892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:11.550534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:11.550807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:11.554300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] PATH_INFO: `/v3/projects/04f189e0d8ea4c3bbb8a5eaea942f3b7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:11.554300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f189e0d8ea4c3bbb8a5eaea942f3b7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:11.554520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:11.680322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f189e0d8ea4c3bbb8a5eaea942f3b7', 'request_id': 'req-3272c537-0605-47f5-a10d-77941e021adf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:11.686179 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f189e0d8ea4c3bbb8a5eaea942f3b7', 'request_id': 'req-3272c537-0605-47f5-a10d-77941e021adf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:11.692202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f189e0d8ea4c3bbb8a5eaea942f3b7', 'request_id': 'req-3272c537-0605-47f5-a10d-77941e021adf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:11.784059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:11.789052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] The token cache is being invalidate because project 04f189e0d8ea4c3bbb8a5eaea942f3b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:11.789388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:11.793978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3272c537-0605-47f5-a10d-77941e021adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:11.793978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1065/2115] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:11 2022] DELETE /identity/v3/projects/04f189e0d8ea4c3bbb8a5eaea942f3b7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:11.796751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:11.797211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:11.797329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:11.816792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:11.817977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:11.877690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1051/2116] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:11.885588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:11.885876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:11.886149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:11.906532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:11.907751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:11.919548 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1066/2117] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:11.923979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:11.936960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55379e-eeb8-4ab0-b6ce-c97ead4e4359 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:11.943178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:11.943931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:11.944347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:11.944693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:11.945816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:11.946204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:11.948784 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:11.964646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1052/2118] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:24:11.969171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:11.978809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0938ff-2bff-4497-a2a2-0cc1b2046c18 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:11.986282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:11.987406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:11.987832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:11.988318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:11.989262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:11.991680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:12.118207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1067/2119] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:11 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:24:12.123539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:12.134102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1b7e5-df2b-4924-a803-2ac643212291 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:12.231314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:12.232108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:12.232497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:12.232872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:12.233685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:12.235741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:12.249412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:12.355244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1053/2120] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:12 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:12.360064 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:12.370936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575231f2-bb53-4979-8972-e65d78d36742 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:12.376176 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:12.376781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:12.377062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:12.377333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:12.377973 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:12.379608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:12.389989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1068/2121] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:12.394152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:12.404407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792a38a-bd95-4a38-9feb-857b60b4a8df admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:12.409219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c63947-1775-49c9-9ce6-642145242e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:12.409806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c63947-1775-49c9-9ce6-642145242e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:12.410087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c63947-1775-49c9-9ce6-642145242e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:12.410359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c63947-1775-49c9-9ce6-642145242e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:12.411026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c63947-1775-49c9-9ce6-642145242e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:12.412654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c63947-1775-49c9-9ce6-642145242e98 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:12.422753 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1054/2122] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:12.426867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:12.437500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfab7cf-019f-4c36-8584-a03afdbc4f00 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:12.442115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:12.442756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:12.443034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:12.443332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] PATH_INFO: `/v3/projects/529c4578d19e426f956fd8a3fe36a358/users/ab5eeb3837104a48a337537473aa7da8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:12.458476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] RBAC: Authorizing `identity:create_grant(project_id=529c4578d19e426f956fd8a3fe36a358, user_id=ab5eeb3837104a48a337537473aa7da8, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:12.461206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:12.560163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1069/2123] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:12 2022] PUT /identity/v3/projects/529c4578d19e426f956fd8a3fe36a358/users/ab5eeb3837104a48a337537473aa7da8/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:12.565836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aeafc-d2c2-48d6-a7ab-4ab4346f9431 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:12.566123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aeafc-d2c2-48d6-a7ab-4ab4346f9431 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:12.566401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5aeafc-d2c2-48d6-a7ab-4ab4346f9431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:12.587289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5d5aeafc-d2c2-48d6-a7ab-4ab4346f9431 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:12.588536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5d5aeafc-d2c2-48d6-a7ab-4ab4346f9431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:12.646655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1055/2124] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:12.656917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:12.658622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:12.714848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4c304-7270-4023-a3b8-1dfc3210949c admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:12.720392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:12.721003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:12.721281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:12.721552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:12.731371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:12.732821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:12.742288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1070/2125] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:14.331634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:14.332195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:14.334322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:14.363819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] MFA Rules not processed for user `ab5eeb3837104a48a337537473aa7da8`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:14.365493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:14.438989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1056/2126] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:14.443791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:14.454646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0804f05-0b67-45cc-a152-7a452616309b service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:14.459110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:14.459700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:14.459975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:14.460270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:14.460895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:14.462785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:14.554554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1071/2127] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:14 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:24:14.558766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:14.581678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412f9249-b0ff-4707-9eb7-dfc6354e9fe0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:14.646381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:14.646995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:14.647297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:14.647571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:14.648232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:14.650601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:14.770406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1057/2128] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:14 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:14.783134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:14.793339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d7a79-e03f-4fcd-9b2f-38ca15a8b596 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:14.797893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:14.798487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:14.798763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:14.799033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:14.799671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:14.801316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:14.813495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1072/2129] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:14.825183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:14.831444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:14.835134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a3a82-5124-42f7-9cb4-f98935ad54ee admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:14.839725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:14.840513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:14.840797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:14.841073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] PATH_INFO: `/v3/projects/cf343e8b3bbc475392d203bab13b8519/users/5782ea019e964ef7a321e96ef2b170ce/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:14.855556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf343e8b3bbc475392d203bab13b8519, user_id=5782ea019e964ef7a321e96ef2b170ce, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:14.858451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:14.981922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1058/2130] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:14 2022] PUT /identity/v3/projects/cf343e8b3bbc475392d203bab13b8519/users/5782ea019e964ef7a321e96ef2b170ce/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:14.985824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:14.995825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a706202-bc05-4988-8f89-1a3d5aac32c4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:15.052070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06404e-2469-44c9-bc4e-d3c0315bfa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:15.052671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06404e-2469-44c9-bc4e-d3c0315bfa48 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:15.052952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06404e-2469-44c9-bc4e-d3c0315bfa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:15.053222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06404e-2469-44c9-bc4e-d3c0315bfa48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:15.053848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06404e-2469-44c9-bc4e-d3c0315bfa48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:15.055515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06404e-2469-44c9-bc4e-d3c0315bfa48 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:15.060970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:15.066044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1073/2131] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:15.070241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:15.081347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804ffd58-5dce-407f-b6aa-db69a1b753c7 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:15.086035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:15.086625 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:15.086907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:15.087178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] PATH_INFO: `/v3/projects/cf343e8b3bbc475392d203bab13b8519/users/5782ea019e964ef7a321e96ef2b170ce/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:15.089380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf343e8b3bbc475392d203bab13b8519, user_id=5782ea019e964ef7a321e96ef2b170ce, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:15.092062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:15.102328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1059/2132] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:15 2022] PUT /identity/v3/projects/cf343e8b3bbc475392d203bab13b8519/users/5782ea019e964ef7a321e96ef2b170ce/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:16.455386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:16.455919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:16.455919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:16.488957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] MFA Rules not processed for user `5782ea019e964ef7a321e96ef2b170ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:16.490498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:16.571285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1074/2133] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:16.584475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:16.587081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:16.647556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0ce25-4f9c-4859-a749-2b5c7ab867ae admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:16.651255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a2ac8-33b3-464b-a229-c23fd4c89016 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:16.651859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a2ac8-33b3-464b-a229-c23fd4c89016 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:16.652172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a2ac8-33b3-464b-a229-c23fd4c89016 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:16.653040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a2ac8-33b3-464b-a229-c23fd4c89016 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:16.713290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a2ac8-33b3-464b-a229-c23fd4c89016 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:16.714642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135a2ac8-33b3-464b-a229-c23fd4c89016 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:16.725411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1060/2134] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:21.688343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:21.698049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a004584-86d3-4015-ac86-6d4ee924638b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:21.702030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:21.702633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:21.702908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:21.703173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:21.712330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:21.713781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:21.723743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1075/2135] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:26.721182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:31.719547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=40 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:35.172498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27a90-55f0-4c19-86de-b95be84815f0 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:35.173000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27a90-55f0-4c19-86de-b95be84815f0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:35.173088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27a90-55f0-4c19-86de-b95be84815f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:35.192709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-21d27a90-55f0-4c19-86de-b95be84815f0 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:35.193886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-21d27a90-55f0-4c19-86de-b95be84815f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:35.247970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1061/2136] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:24:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:35.290727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:35.292183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:35.361023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cda230-10ca-43fa-bc19-210a7807733f service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:35.365971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:35.366614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:35.366886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:35.367151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:35.376526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:35.378094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:35.385998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1076/2137] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:24:36.882116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082484-4b51-4138-994f-c10046c4df81 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:36.882597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082484-4b51-4138-994f-c10046c4df81 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:36.882690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082484-4b51-4138-994f-c10046c4df81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:36.906500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-66082484-4b51-4138-994f-c10046c4df81 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:36.906500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-66082484-4b51-4138-994f-c10046c4df81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:36.913536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1062/2138] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 10 11:24:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:36.927178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:36.940463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0fd6b-0e5d-42b3-8eaf-16e58b93d707 service swift] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:36.945405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729ed07-7f12-4a6b-8d5f-8c4613ddd5e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:36.946014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729ed07-7f12-4a6b-8d5f-8c4613ddd5e5 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:36.946320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729ed07-7f12-4a6b-8d5f-8c4613ddd5e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:36.946591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729ed07-7f12-4a6b-8d5f-8c4613ddd5e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:36.956809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729ed07-7f12-4a6b-8d5f-8c4613ddd5e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:36.958446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729ed07-7f12-4a6b-8d5f-8c4613ddd5e5 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:36.967450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1077/2139] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:24:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:24:37.401275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66082484-4b51-4138-994f-c10046c4df81 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:37.403730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-66082484-4b51-4138-994f-c10046c4df81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:37.482786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66082484-4b51-4138-994f-c10046c4df81 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:37.487377 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:37.488022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:37.488313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:37.488582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] PATH_INFO: `/v3/users/c51993c317d347b0939140d2f520827a` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:37.498929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51993c317d347b0939140d2f520827a)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:37.498929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:37.929960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51993c317d347b0939140d2f520827a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:37.930730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51993c317d347b0939140d2f520827a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:37.931451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51993c317d347b0939140d2f520827a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:37.939938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51993c317d347b0939140d2f520827a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:37.979033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:37.979581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:37.979759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:38.002500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:38.006222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:38.012106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1078/2140] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:24:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:38.024571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:38.025981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:38.042099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13535805-1e2f-4d92-b900-7dfacce42910 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:38.048254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f7f47-6bbd-452a-982d-609249f42e60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:38.048881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7f47-6bbd-452a-982d-609249f42e60 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:38.049170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7f47-6bbd-452a-982d-609249f42e60 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:38.049444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f7f47-6bbd-452a-982d-609249f42e60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:38.059025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f7f47-6bbd-452a-982d-609249f42e60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:38.061005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f7f47-6bbd-452a-982d-609249f42e60 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:38.073286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1079/2141] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:24:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:24:38.159633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51993c317d347b0939140d2f520827a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:38.178285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1063/2142] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:37 2022] DELETE /identity/v3/users/c51993c317d347b0939140d2f520827a => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:38.182960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca70d3f-9cc6-4728-a6c6-3de2c55cf3a6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:38.183185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca70d3f-9cc6-4728-a6c6-3de2c55cf3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:38.183542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca70d3f-9cc6-4728-a6c6-3de2c55cf3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:38.203047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0ca70d3f-9cc6-4728-a6c6-3de2c55cf3a6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:38.204273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0ca70d3f-9cc6-4728-a6c6-3de2c55cf3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:38.270604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1080/2143] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:38.280542 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:38.281966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:38.303762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:38.342155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1bfc8-0d2b-49a3-aa66-cdfa29619146 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:38.345882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:38.348234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:38.348234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:38.348234 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:38.357297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:38.358743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:38.368670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1064/2144] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:38.864822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:38.865109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:38.865381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:38.885008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:38.886191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:38.922273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:38.923698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:38.937365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3791bb74-35fd-4833-924b-962109072d28 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:38.943695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:38.944388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:38.944827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:38.945146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] PATH_INFO: `/v3/projects/b86b9874f91c4a16b567e035eb966362` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:38.946275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b86b9874f91c4a16b567e035eb966362)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:38.946835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1081/2145] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 10 11:24:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:38.948538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:38.963209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:38.968311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:39.031923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a3a10-758f-43c4-afd7-d1a15d2b4f42 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.037255 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8fb53-2e6b-4f68-a029-c2cfebe9ce19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.037860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8fb53-2e6b-4f68-a029-c2cfebe9ce19 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.038136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8fb53-2e6b-4f68-a029-c2cfebe9ce19 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.038480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8fb53-2e6b-4f68-a029-c2cfebe9ce19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.046883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8fb53-2e6b-4f68-a029-c2cfebe9ce19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.048616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8fb53-2e6b-4f68-a029-c2cfebe9ce19 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.056433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1082/2146] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:24:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:24:39.102652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b86b9874f91c4a16b567e035eb966362', 'request_id': 'req-f204fa2d-6627-4dc3-a91a-30633a7b5b60'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:39.109145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b86b9874f91c4a16b567e035eb966362', 'request_id': 'req-f204fa2d-6627-4dc3-a91a-30633a7b5b60'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:39.114907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b86b9874f91c4a16b567e035eb966362', 'request_id': 'req-f204fa2d-6627-4dc3-a91a-30633a7b5b60'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:39.193741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] The token cache is being invalidate because project b86b9874f91c4a16b567e035eb966362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:39.194029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:39.194642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f204fa2d-6627-4dc3-a91a-30633a7b5b60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:39.196271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1065/2147] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:38 2022] DELETE /identity/v3/projects/b86b9874f91c4a16b567e035eb966362 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:39.202217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.202498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.202805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.222218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:39.223406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:39.285125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1083/2148] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:39.293687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.293687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.297551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.314046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:39.315295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:39.323329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1066/2149] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:39.329984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:39.339679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a748cc-cc38-4466-be1f-49f34840bffe None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.345073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.345768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.350312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.350312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.350312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:39.350312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.350312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.362800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1084/2150] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:24:39.375303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:39.389840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0218cf-5573-42b5-867e-468723e5ebe6 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.394831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.395436 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.395716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.395988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.396701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.398635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.509747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1067/2151] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:39 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:39.520946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:39.526875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10546526-a09c-41fa-940c-8a62a6bf4551 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.593729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.594407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.594689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.594963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.595600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.597541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.722104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1085/2152] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:39 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:39.726690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:39.740257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d544950-1dd0-4fc0-a1db-5f04683eadfa admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.745426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.746191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.752340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.752340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.752340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.752340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.762239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1068/2153] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:39.767920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:39.779954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06564d20-36ed-480b-b3ef-66b57b9a79ed admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.787085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4258d20-66cb-4be8-9aed-8bdb0c99a0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.788133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4258d20-66cb-4be8-9aed-8bdb0c99a0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.788467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4258d20-66cb-4be8-9aed-8bdb0c99a0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.788798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4258d20-66cb-4be8-9aed-8bdb0c99a0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.789503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4258d20-66cb-4be8-9aed-8bdb0c99a0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.792414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4258d20-66cb-4be8-9aed-8bdb0c99a0ad admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.795733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:39.805587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1086/2154] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:39.813019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:39.826079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e71107a-0068-4725-b6e8-545020eb3be7 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:39.831522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:39.832702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.832835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.833258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] PATH_INFO: `/v3/projects/e05759c8ab0248e58d35630a611912f0/users/46dffe5f71cf4d2ca7bf42f86d2bb07a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:39.851403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e05759c8ab0248e58d35630a611912f0, user_id=46dffe5f71cf4d2ca7bf42f86d2bb07a, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:39.853992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:39.992186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1069/2155] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:39 2022] PUT /identity/v3/projects/e05759c8ab0248e58d35630a611912f0/users/46dffe5f71cf4d2ca7bf42f86d2bb07a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:39.997371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:39.997682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:39.997953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:40.019656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:40.019656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:40.077201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1087/2156] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:40.086975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:40.088696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:40.132215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:40.146261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bc3bb-24c6-48ad-a1f6-56d35c77592d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:40.149936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea490f0-df14-4dc8-b54a-73f0c0ce4962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:40.150540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea490f0-df14-4dc8-b54a-73f0c0ce4962 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:40.150818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea490f0-df14-4dc8-b54a-73f0c0ce4962 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:40.151123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea490f0-df14-4dc8-b54a-73f0c0ce4962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:40.160468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea490f0-df14-4dc8-b54a-73f0c0ce4962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:40.162890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea490f0-df14-4dc8-b54a-73f0c0ce4962 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:40.173573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1070/2157] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:43.814220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:43.817008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:43.830924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa0c33-54bb-43a3-a8ea-d583a773da59 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:43.850722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:43.850722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:43.850722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:43.850722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] PATH_INFO: `/v3/users/a1c55eb65703476994509add9731e0db` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:43.851010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c55eb65703476994509add9731e0db)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:43.851010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:44.122704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c55eb65703476994509add9731e0db'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:44.122704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c55eb65703476994509add9731e0db'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:44.122704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c55eb65703476994509add9731e0db'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:44.129733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c55eb65703476994509add9731e0db'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:44.240411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c55eb65703476994509add9731e0db'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:44.267303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1088/2158] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:43 2022] DELETE /identity/v3/users/a1c55eb65703476994509add9731e0db => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:44.272338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:44.272844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:44.273073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:44.305262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:44.307702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:44.377349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1071/2159] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:44.387957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:44.389452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:44.454705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a3b93-ea50-4797-9e41-68bc7c0e7904 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:44.459607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727cbd6-1c12-49bd-a5d3-523dccd5100a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:44.460290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727cbd6-1c12-49bd-a5d3-523dccd5100a service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:44.460524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727cbd6-1c12-49bd-a5d3-523dccd5100a service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:44.460807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727cbd6-1c12-49bd-a5d3-523dccd5100a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:44.470671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727cbd6-1c12-49bd-a5d3-523dccd5100a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:44.472144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727cbd6-1c12-49bd-a5d3-523dccd5100a service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:44.483770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1089/2160] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:44.879752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:44.881504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:44.909892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7896517-ebbb-42db-804f-8a4be1eaa4e3 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:44.916524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:44.917465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:44.919442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:44.920883 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] PATH_INFO: `/v3/projects/29447f0b81cb4a8bbeab875115c2b542` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:44.922543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29447f0b81cb4a8bbeab875115c2b542)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:44.925369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:45.094749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29447f0b81cb4a8bbeab875115c2b542', 'request_id': 'req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:45.101950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29447f0b81cb4a8bbeab875115c2b542', 'request_id': 'req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:45.109107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29447f0b81cb4a8bbeab875115c2b542', 'request_id': 'req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:45.224586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] The token cache is being invalidate because project 29447f0b81cb4a8bbeab875115c2b542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:45.224586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:45.224586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-cffd0436-d7e3-49ec-bb9f-0e4b001b9ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:45.230130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1072/2161] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:44 2022] DELETE /identity/v3/projects/29447f0b81cb4a8bbeab875115c2b542 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:45.231749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.232330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.232490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.252609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:45.253822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:45.284551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:45.343284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1090/2162] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:45.352214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.352874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.353380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.378400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:45.379826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:45.395014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1073/2163] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:45.399838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:45.415797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f327910-b319-4f62-b486-16c16b1f80f6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:45.421730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:45.422353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.422625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.422889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.423533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:24:45.423835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:45.427372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:45.463243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1091/2164] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:24:45.464325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:45.476462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dba0d-aa9d-47ec-b540-569baf9c3859 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:45.486897 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:45.488338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.488338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.490370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.490370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:45.493285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:45.611048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1074/2165] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:45 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:24:45.612343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:45.623134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c53f8-f7c5-4df6-a7fa-a3c24849de01 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:45.680546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:45.681166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.681444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.681714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.682343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:45.685080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:45.815285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1092/2166] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:45 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:45.816826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:45.833341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4acaf5f-a6f1-4ec4-98a4-2827a78d8308 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:45.839274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:45.840862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.841284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.841670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.842404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:45.844176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:45.862608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1075/2167] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:45.865228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:45.884388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59778bbf-0d72-48e9-b1f2-f86c66b66dd2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:45.896430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa28362-5eeb-4ec1-9819-ff2f766cf532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:45.897297 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa28362-5eeb-4ec1-9819-ff2f766cf532 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.898033 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa28362-5eeb-4ec1-9819-ff2f766cf532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.898320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa28362-5eeb-4ec1-9819-ff2f766cf532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.899031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa28362-5eeb-4ec1-9819-ff2f766cf532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:45.900673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa28362-5eeb-4ec1-9819-ff2f766cf532 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:45.928831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1093/2168] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:45 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:45.929412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:45.943588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7908-23c4-4a7b-843f-bd0ecde10292 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:45.951387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:45.952160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:45.952373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:45.959458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] PATH_INFO: `/v3/projects/e202fe4250104681a9fe73fc4750c33f/users/e49a2b5153b04661b3ef6fe699867005/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:45.986382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e202fe4250104681a9fe73fc4750c33f, user_id=e49a2b5153b04661b3ef6fe699867005, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:45.989087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:46.149229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1076/2169] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:45 2022] PUT /identity/v3/projects/e202fe4250104681a9fe73fc4750c33f/users/e49a2b5153b04661b3ef6fe699867005/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:46.154343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c713801-54f4-4415-927a-d0e34e2e9699 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:46.154627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c713801-54f4-4415-927a-d0e34e2e9699 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:46.154900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c713801-54f4-4415-927a-d0e34e2e9699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:46.175514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8c713801-54f4-4415-927a-d0e34e2e9699 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:46.177256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8c713801-54f4-4415-927a-d0e34e2e9699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:46.281401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1094/2170] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:46.303595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:46.305371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:46.311871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:46.373318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ad33bc-63c5-4147-8ad8-06465d534edd admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:46.377063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:46.377680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:46.377961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:46.378231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:46.389346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:46.390718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:46.400835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1077/2171] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:47.036627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.037090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.037542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.071624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] MFA Rules not processed for user `46dffe5f71cf4d2ca7bf42f86d2bb07a`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:47.072953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:47.153287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1095/2172] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:47.157413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:47.158963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:47.172581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36198cef-06da-44f0-8855-f944d7125c7d service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:47.179066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:47.179901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.180212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.180488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.181127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:47.183022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:47.294163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1078/2173] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:47 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:47.298563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:47.309750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eab279-51d2-4c11-bcfa-b6911ab06ffe None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:47.329148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:47.389860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:47.391063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.391349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.391655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.392416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:47.394331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:47.524497 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:47.525020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1096/2174] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:47 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:47.542420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d8218-93a2-4e8e-83c0-3e07e130254c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:47.547859 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:47.548514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.548797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.549070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.549723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:47.551390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:47.564235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1079/2175] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:24:47.574467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:47.585811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb4498-d621-4014-a46b-5cce6e8f6499 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:47.591803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:47.592690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.592966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.593231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] PATH_INFO: `/v3/projects/dbd2f7d25d2643b1a008cf3d07822038/users/8ab73f35559a4e908e15e7ae8aa5dd51/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.609969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd2f7d25d2643b1a008cf3d07822038, user_id=8ab73f35559a4e908e15e7ae8aa5dd51, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:47.612839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:47.805881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1097/2176] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:47 2022] PUT /identity/v3/projects/dbd2f7d25d2643b1a008cf3d07822038/users/8ab73f35559a4e908e15e7ae8aa5dd51/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:47.809861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:47.819717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551a649a-e53d-4b5b-9470-873a9bf0dd0e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:47.885851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb90c99-d65e-462c-bd86-a0f7b63b128c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:47.887052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb90c99-d65e-462c-bd86-a0f7b63b128c admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.887665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb90c99-d65e-462c-bd86-a0f7b63b128c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.887665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb90c99-d65e-462c-bd86-a0f7b63b128c admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.888187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb90c99-d65e-462c-bd86-a0f7b63b128c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:47.889790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb90c99-d65e-462c-bd86-a0f7b63b128c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:47.899587 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1080/2177] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:47.904655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:47.915309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49243546-36bc-4a48-a2dd-2ea33e900bb4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:47.920269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:47.921028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:47.921409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:47.921786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] PATH_INFO: `/v3/projects/dbd2f7d25d2643b1a008cf3d07822038/users/8ab73f35559a4e908e15e7ae8aa5dd51/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:47.923781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd2f7d25d2643b1a008cf3d07822038, user_id=8ab73f35559a4e908e15e7ae8aa5dd51, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:47.926495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:47.937703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1098/2178] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:47 2022] PUT /identity/v3/projects/dbd2f7d25d2643b1a008cf3d07822038/users/8ab73f35559a4e908e15e7ae8aa5dd51/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:54.800348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:54.800868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:54.800868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:54.838330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] MFA Rules not processed for user `e49a2b5153b04661b3ef6fe699867005`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:54.840319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:54.976878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:54.990636 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1081/2179] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:54.993054 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:55.163855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606d084-4908-4886-bbf5-a2292e4e72f7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:55.171930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:55.172734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:55.180935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:55.180935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:55.180935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:55.180935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:55.217025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:55.308715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1099/2180] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:24:54 2022] POST /identity/v3/projects => generated 383 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:55.312362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:55.331241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca533236-9527-42a8-9345-8e0dcf219640 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:55.485854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:55.487181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:55.487646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:55.488215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:55.489090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:55.492040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:55.599847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1082/2181] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:24:55 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:24:55.603772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:55.616631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa34c69-1d35-46c8-99e6-cce7f83e9f2c admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:55.624733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:55.625521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:55.625908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:55.626353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:55.627111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:55.629199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:55.653321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1100/2182] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:24:55.654887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:55.690878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b7457-f53f-44a8-a300-f10f5f8b7125 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:55.696660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:55.697632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:55.697924 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:55.698198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] PATH_INFO: `/v3/projects/380e0d1a6e0745e8b4befee713665c18/users/1b378d9208e9456dac3744ebecd4ae97/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:55.730105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=380e0d1a6e0745e8b4befee713665c18, user_id=1b378d9208e9456dac3744ebecd4ae97, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:55.734880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:55.869208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1083/2183] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:55 2022] PUT /identity/v3/projects/380e0d1a6e0745e8b4befee713665c18/users/1b378d9208e9456dac3744ebecd4ae97/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:55.873498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:55.883574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83836ea-d648-494d-a272-319a1fe4eb76 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:55.941487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76546fac-5b9b-41c8-97cc-fe6bc704c235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:55.943163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76546fac-5b9b-41c8-97cc-fe6bc704c235 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:55.943163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76546fac-5b9b-41c8-97cc-fe6bc704c235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:55.943163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76546fac-5b9b-41c8-97cc-fe6bc704c235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:55.943407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76546fac-5b9b-41c8-97cc-fe6bc704c235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:55.944853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76546fac-5b9b-41c8-97cc-fe6bc704c235 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:55.955433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1101/2184] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:24:55.959745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:55.970637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3fe19-02a0-4e32-94f4-424702dbc542 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:55.975942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:55.976356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:55.976634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:55.976906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] PATH_INFO: `/v3/projects/380e0d1a6e0745e8b4befee713665c18/users/1b378d9208e9456dac3744ebecd4ae97/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:55.978847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=380e0d1a6e0745e8b4befee713665c18, user_id=1b378d9208e9456dac3744ebecd4ae97, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:55.982112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:55.992458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1084/2185] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:24:55 2022] PUT /identity/v3/projects/380e0d1a6e0745e8b4befee713665c18/users/1b378d9208e9456dac3744ebecd4ae97/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:56.765890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396949d6-fa2e-4227-b102-eda4f6bde9de None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:56.766407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396949d6-fa2e-4227-b102-eda4f6bde9de None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:56.766407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396949d6-fa2e-4227-b102-eda4f6bde9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:56.804042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-396949d6-fa2e-4227-b102-eda4f6bde9de None None] MFA Rules not processed for user `8ab73f35559a4e908e15e7ae8aa5dd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:56.805307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-396949d6-fa2e-4227-b102-eda4f6bde9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:56.894423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1102/2186] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:56.908530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:56.911072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:57.049761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706e6db-f690-4983-ab74-0bcfc7949631 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:57.057385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:57.058182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:57.058538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:57.059081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:57.111914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41268dfc-bc50-4c84-9fde-db7c29971e00 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:57.112519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41268dfc-bc50-4c84-9fde-db7c29971e00 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:57.113558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41268dfc-bc50-4c84-9fde-db7c29971e00 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:57.115754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1103/2187] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:24:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:24:57.118429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:57.131959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:57.133758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:57.143945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1085/2188] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:24:57.155100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:57.155444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:57.155924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:57.176923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] MFA Rules not processed for user `8593dfcc1d0542bfbaa6fb169b01ab90`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:57.178140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:57.235681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:57.251460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1104/2189] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:24:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:24:57.267675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:57.269177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:57.284044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ab5eb1-79ea-46e7-bb07-c5f966e5eb77 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:57.290060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:57.290842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:57.291190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:57.291525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:57.309961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:57.312428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:57.313638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:57.315306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:57.335171 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1086/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:57.410706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55feb8-acf7-44c2-8ab2-a9cd8fa1e4f1 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:57.422209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:57.422209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:57.422209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:57.422209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] PATH_INFO: `/v3/users/ab5eeb3837104a48a337537473aa7da8` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:57.422759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5eeb3837104a48a337537473aa7da8)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:57.422759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:57.565688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:57.567421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:57.587709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aca1e2-7c22-4c76-bee9-8ffdfbe7f5c6 service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:57.595030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:57.595682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:57.595961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:57.596304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] PATH_INFO: `/v3/users/20336c8417b24938a4b5aca538d72727` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:57.597673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20336c8417b24938a4b5aca538d72727)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:57.600365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:57.763163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5eeb3837104a48a337537473aa7da8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:57.763880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5eeb3837104a48a337537473aa7da8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:57.764526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5eeb3837104a48a337537473aa7da8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:57.771626 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5eeb3837104a48a337537473aa7da8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.017172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-bd97ec9d-049c-48c8-a526-6bc05465e082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5eeb3837104a48a337537473aa7da8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.035908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1105/2191] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:57 2022] DELETE /identity/v3/users/ab5eeb3837104a48a337537473aa7da8 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:58.147799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20336c8417b24938a4b5aca538d72727'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.149358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20336c8417b24938a4b5aca538d72727'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.153046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20336c8417b24938a4b5aca538d72727'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.180645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20336c8417b24938a4b5aca538d72727'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.290047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20336c8417b24938a4b5aca538d72727'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.318989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1087/2192] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:57 2022] DELETE /identity/v3/users/20336c8417b24938a4b5aca538d72727 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:58.323560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:58.323953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:58.324377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:58.345928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:24:58.347865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:58.406677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:58.408240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:58.432371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1106/2193] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:24:58.455356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:58.457324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:58.532315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa71c8-db57-478d-9f31-71856ac1531e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:58.541165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:58.543798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:58.548982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:58.548982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] PATH_INFO: `/v3/projects/529c4578d19e426f956fd8a3fe36a358` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:58.548982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] RBAC: Authorizing `identity:delete_project(project_id=529c4578d19e426f956fd8a3fe36a358)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:58.549296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a0746-597d-47d2-a0c2-65023b5a1244 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:58.549928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:58.551067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:58.551858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:58.552253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:58.553063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:58.562908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:58.564624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:58.576229 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1107/2194] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:24:58.678857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '529c4578d19e426f956fd8a3fe36a358', 'request_id': 'req-e9c02780-0d1f-4721-b376-ab7659d5194d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.685712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '529c4578d19e426f956fd8a3fe36a358', 'request_id': 'req-e9c02780-0d1f-4721-b376-ab7659d5194d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.692372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '529c4578d19e426f956fd8a3fe36a358', 'request_id': 'req-e9c02780-0d1f-4721-b376-ab7659d5194d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.816986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] The token cache is being invalidate because project 529c4578d19e426f956fd8a3fe36a358 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:24:58.833160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.833160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:58.833160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1088/2195] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:58 2022] DELETE /identity/v3/projects/529c4578d19e426f956fd8a3fe36a358 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:24:58.833588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:58.837565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492b51e-ab12-44d4-9dd6-544439304446 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:58.913958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:58.913958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:58.913958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:58.913958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] PATH_INFO: `/v3/users/5782ea019e964ef7a321e96ef2b170ce` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:58.914407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5782ea019e964ef7a321e96ef2b170ce)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:58.914407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:59.330421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5782ea019e964ef7a321e96ef2b170ce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.331473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5782ea019e964ef7a321e96ef2b170ce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.332327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5782ea019e964ef7a321e96ef2b170ce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.340935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5782ea019e964ef7a321e96ef2b170ce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.462736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:59.470216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-22027bc9-c90b-4465-ade7-2c592de65a13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5782ea019e964ef7a321e96ef2b170ce'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.517827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:24:59.517983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1108/2196] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:24:58 2022] DELETE /identity/v3/users/5782ea019e964ef7a321e96ef2b170ce => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:24:59.521781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:24:59.606208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c02780-0d1f-4721-b376-ab7659d5194d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:24:59.753489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:24:59.754308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:24:59.754591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:24:59.755275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] PATH_INFO: `/v3/projects/62a0053b8c614c36b30d71654e2b3ad0` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:24:59.756453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a0053b8c614c36b30d71654e2b3ad0)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:24:59.760354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:24:59.766642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:24:59.973343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a0053b8c614c36b30d71654e2b3ad0', 'request_id': 'req-6065af40-1bc9-49c3-8295-b73dcc984fd0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.983787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a0053b8c614c36b30d71654e2b3ad0', 'request_id': 'req-6065af40-1bc9-49c3-8295-b73dcc984fd0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:24:59.990395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a0053b8c614c36b30d71654e2b3ad0', 'request_id': 'req-6065af40-1bc9-49c3-8295-b73dcc984fd0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:00.110716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] The token cache is being invalidate because project 62a0053b8c614c36b30d71654e2b3ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:00.111162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:00.112988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6065af40-1bc9-49c3-8295-b73dcc984fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:00.115208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1089/2197] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:24:59 2022] DELETE /identity/v3/projects/62a0053b8c614c36b30d71654e2b3ad0 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:00.119733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cd12b-2567-4406-adc2-019719589179 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.120267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cd12b-2567-4406-adc2-019719589179 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.120347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11cd12b-2567-4406-adc2-019719589179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.157362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d11cd12b-2567-4406-adc2-019719589179 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:00.158910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d11cd12b-2567-4406-adc2-019719589179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:00.270583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1109/2198] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:00.279590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.279920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.280193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.322346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:00.324049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:00.343844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1090/2199] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:00.348186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cd12b-2567-4406-adc2-019719589179 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:00.362783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11cd12b-2567-4406-adc2-019719589179 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:00.374542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:00.375186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.375465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.375737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.376437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:00.376739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:00.378681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:00.413432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1110/2200] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:00.414546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:00.432358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9063972-83ac-47de-9a7c-e00439f7c24e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:00.438479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:00.439411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.439914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.440416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.441284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:00.443426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:00.650631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1091/2201] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:00 2022] POST /identity/v3/projects => generated 363 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:00.655556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:00.665768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b84d-5496-4102-8397-5bd5aab3c7df admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:00.707173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:00.713351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:00.789624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:00.790604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.793091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.797428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.799363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:00.810228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93a7b2-168a-4ed8-a69f-b40e35ed5f08 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:00.820517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:00.829615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:00.830643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.832099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.832540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] PATH_INFO: `/v3/projects/cf343e8b3bbc475392d203bab13b8519` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.835895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf343e8b3bbc475392d203bab13b8519)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:00.840340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:00.937647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1111/2202] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:00 2022] POST /identity/v3/users => generated 282 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:00.941502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:00.964556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815575a9-a0f3-40f6-8df0-c6f659a56513 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:00.972174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:00.973145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:00.973797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:00.974283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:00.975240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:00.977278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:01.012086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1112/2203] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:01.015963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:01.029646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ccb17-f882-4672-9435-dc4e2eccfe90 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:01.046302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:01.056124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.056124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.056124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.056124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:01.056124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:01.063828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1113/2204] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:01.067733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:01.121592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16a4e5-60c4-44af-a5fd-71329fefe7f6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:01.136468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf343e8b3bbc475392d203bab13b8519', 'request_id': 'req-10798546-812a-412a-be9a-cd825d18b65d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:01.151698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf343e8b3bbc475392d203bab13b8519', 'request_id': 'req-10798546-812a-412a-be9a-cd825d18b65d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:01.162271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf343e8b3bbc475392d203bab13b8519', 'request_id': 'req-10798546-812a-412a-be9a-cd825d18b65d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:01.204918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491837-631f-4206-958a-f3407d5778f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:01.205619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491837-631f-4206-958a-f3407d5778f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.205894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491837-631f-4206-958a-f3407d5778f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.206178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491837-631f-4206-958a-f3407d5778f2 admin admin] PATH_INFO: `/v3/projects/ba121949bf75417ebaf0aaf183f897bb/users/eea9707b4ca04ed190b6d5eca313a93d/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.241782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491837-631f-4206-958a-f3407d5778f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba121949bf75417ebaf0aaf183f897bb, user_id=eea9707b4ca04ed190b6d5eca313a93d, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:01.244720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f491837-631f-4206-958a-f3407d5778f2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:01.316945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] The token cache is being invalidate because project cf343e8b3bbc475392d203bab13b8519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:01.316945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:01.318170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-10798546-812a-412a-be9a-cd825d18b65d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:01.319865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1092/2205] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:00 2022] DELETE /identity/v3/projects/cf343e8b3bbc475392d203bab13b8519 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:01.325641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bead4a-fc67-435e-b6b1-ea5915d7ab03 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.325932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bead4a-fc67-435e-b6b1-ea5915d7ab03 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.326343 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bead4a-fc67-435e-b6b1-ea5915d7ab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.381222 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-68bead4a-fc67-435e-b6b1-ea5915d7ab03 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:01.388455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-68bead4a-fc67-435e-b6b1-ea5915d7ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:01.391646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1114/2206] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:01 2022] PUT /identity/v3/projects/ba121949bf75417ebaf0aaf183f897bb/users/eea9707b4ca04ed190b6d5eca313a93d/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:01.402616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.403216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.403713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.505938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] MFA Rules not processed for user `eea9707b4ca04ed190b6d5eca313a93d`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:01.509970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:01.696469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1093/2207] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:01.699358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.700411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.700411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.707433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1115/2208] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:01.723533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:01.726367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:01.751170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:01.753470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:01.775487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1094/2209] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:01.779869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:01.856558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb27e1e-737b-4b24-957f-51fb30c0dc0d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:01.856558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:01.856558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.856558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.856900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.856900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:01.856900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:01.856900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:01.857198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bfeae4-b52d-4cfa-98b7-894ddfdef224 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:01.857285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1095/2210] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:25:01.857285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:01.857436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:01.857436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.857436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.857436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.888967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71295a-9bd7-49d0-8c91-b6fda2d7c2b4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:01.918406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831df0e-53a5-4160-808d-cee0d142a263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:01.918406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831df0e-53a5-4160-808d-cee0d142a263 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:01.918866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:01.918866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:01.919014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831df0e-53a5-4160-808d-cee0d142a263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:01.919014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831df0e-53a5-4160-808d-cee0d142a263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:01.919014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831df0e-53a5-4160-808d-cee0d142a263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:01.919495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831df0e-53a5-4160-808d-cee0d142a263 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:01.963343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1116/2211] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:02.396672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1096/2212] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:01 2022] POST /identity/v3/projects => generated 371 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:02.399856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:02.434611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ddfb42-8940-4056-a16d-5eb67e00d6fd service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:02.516907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:02.522473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:02.522473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:02.558879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] MFA Rules not processed for user `46dffe5f71cf4d2ca7bf42f86d2bb07a`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:02.560262 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:02.578441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:02.580521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:02.580810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:02.581420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:02.582767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:02.590259 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:02.646940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1097/2213] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:02.781265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1117/2214] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:02 2022] POST /identity/v3/users => generated 286 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:02.783060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:02.795213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8908e514-adcd-4494-ba2d-d7fc831abeab None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:02.803771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:02.805107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:02.805561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:02.806031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:02.807143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:02.813920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:02.828834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1098/2215] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:02.833183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:02.857311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5e6e-402e-4caa-a5bd-95f77a690399 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:02.865068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62316-9995-49cf-b34f-4c48ac6fb878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:02.865702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62316-9995-49cf-b34f-4c48ac6fb878 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:02.865980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62316-9995-49cf-b34f-4c48ac6fb878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:02.866500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62316-9995-49cf-b34f-4c48ac6fb878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:02.866979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62316-9995-49cf-b34f-4c48ac6fb878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:02.868646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62316-9995-49cf-b34f-4c48ac6fb878 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:02.896109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1118/2216] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:02.898440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:02.913472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348521fe-5680-4782-b401-101d3ce36fe7 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:02.926945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:02.927736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:02.928128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:02.928467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] PATH_INFO: `/v3/projects/3676c9e8e3c7431b93250539f5a903ed/users/5dfdac18a80d4d6fbab774e4e1b4fe48/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:02.956758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3676c9e8e3c7431b93250539f5a903ed, user_id=5dfdac18a80d4d6fbab774e4e1b4fe48, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:02.960457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:02.969546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:03.131710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1099/2217] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:02 2022] PUT /identity/v3/projects/3676c9e8e3c7431b93250539f5a903ed/users/5dfdac18a80d4d6fbab774e4e1b4fe48/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:25:03.151010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:03.151991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:03.151991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:03.255613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] MFA Rules not processed for user `5dfdac18a80d4d6fbab774e4e1b4fe48`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:03.258074 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:03.410855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1119/2218] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:03.425100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:03.510294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f85f0b-0aad-4ac5-b9c6-eb8b5e84c4d4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:03.527674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e27e1b-500c-4a28-a641-45ec800859b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:03.527674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e27e1b-500c-4a28-a641-45ec800859b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:03.527674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e27e1b-500c-4a28-a641-45ec800859b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:03.527674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e27e1b-500c-4a28-a641-45ec800859b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:03.535242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e27e1b-500c-4a28-a641-45ec800859b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:03.537975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e27e1b-500c-4a28-a641-45ec800859b5 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:03.549501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1100/2219] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:03.581172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:03.652484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:03.685854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da848c7-b4c2-49de-b50e-ff3bd7c89ace None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:03.691086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:03.691839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:03.692202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:03.692487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] PATH_INFO: `/v3/users/5dfdac18a80d4d6fbab774e4e1b4fe48` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:03.693912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dfdac18a80d4d6fbab774e4e1b4fe48)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:03.708480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:04.104885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dfdac18a80d4d6fbab774e4e1b4fe48'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:04.107254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dfdac18a80d4d6fbab774e4e1b4fe48'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:04.115580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dfdac18a80d4d6fbab774e4e1b4fe48'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:04.130666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dfdac18a80d4d6fbab774e4e1b4fe48'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:04.208694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dfdac18a80d4d6fbab774e4e1b4fe48'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:04.252722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1120/2220] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:25:03 2022] DELETE /identity/v3/users/5dfdac18a80d4d6fbab774e4e1b4fe48 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:04.257195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:04.257635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:04.257635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:04.290538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:04.292152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:04.366130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1101/2221] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:04.377541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:04.379419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:04.460946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f67fd25-013f-42c1-8664-cb553daa27bd admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:04.465518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbaab69-18e5-4978-9867-a596569cec3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:04.466476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaab69-18e5-4978-9867-a596569cec3f service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:04.466921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaab69-18e5-4978-9867-a596569cec3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:04.467319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbaab69-18e5-4978-9867-a596569cec3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:04.482912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbaab69-18e5-4978-9867-a596569cec3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:04.484646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbaab69-18e5-4978-9867-a596569cec3f service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:04.498121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1121/2222] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:05.327307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:05.337916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:05.400054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540bd7af-bca6-4d2c-af8f-0ec8b81a8016 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:05.424418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:05.425755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:05.426676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:05.427174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] PATH_INFO: `/v3/projects/3676c9e8e3c7431b93250539f5a903ed` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:05.429087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3676c9e8e3c7431b93250539f5a903ed)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:05.431484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:05.594261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:05.595886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:05.596095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:05.646063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3676c9e8e3c7431b93250539f5a903ed', 'request_id': 'req-ef723603-6a19-410d-b64a-e6323c2bb841'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:05.647449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] MFA Rules not processed for user `1b378d9208e9456dac3744ebecd4ae97`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:05.648874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:05.654194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3676c9e8e3c7431b93250539f5a903ed', 'request_id': 'req-ef723603-6a19-410d-b64a-e6323c2bb841'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:05.667157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3676c9e8e3c7431b93250539f5a903ed', 'request_id': 'req-ef723603-6a19-410d-b64a-e6323c2bb841'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:05.729904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1122/2223] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:05.743002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:05.746125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:05.772270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] The token cache is being invalidate because project 3676c9e8e3c7431b93250539f5a903ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:25:05.772568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:05.783860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef723603-6a19-410d-b64a-e6323c2bb841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:25:05.783860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1102/2224] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:25:05 2022] DELETE /identity/v3/projects/3676c9e8e3c7431b93250539f5a903ed => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:05.783860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bd977-3220-4d4e-9f45-7a569f8a1acd None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:05.783860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bd977-3220-4d4e-9f45-7a569f8a1acd None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:05.783860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7bd977-3220-4d4e-9f45-7a569f8a1acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:05.818659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4e7bd977-3220-4d4e-9f45-7a569f8a1acd None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:05.818659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4e7bd977-3220-4d4e-9f45-7a569f8a1acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:05.818879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0aaa6-1ad7-4509-ac3d-fc1347a0d8d9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:05.822056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:05.822683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:05.822965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:05.823232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:05.825911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:05.915733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1103/2225] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:05.924158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:05.924444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:05.924717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:05.924844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:05.926468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:05.936943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1123/2226] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:05.946718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:05.947979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:05.957154 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1104/2227] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:05.967305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:05.977905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbefde-7908-465e-b52e-1a5aa050bce7 service glance] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:05.983309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:05.983910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:05.984334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:05.984607 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:05.985249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:25:05.985552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:05.987207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:05.999942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1124/2228] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:25:06.004503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.014661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea15df-55f8-4343-8b8d-f91f0c1add93 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.022889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c47959-efe2-4aa3-9975-6451d29ae1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.022889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c47959-efe2-4aa3-9975-6451d29ae1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.022889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c47959-efe2-4aa3-9975-6451d29ae1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.022889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c47959-efe2-4aa3-9975-6451d29ae1a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.023257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c47959-efe2-4aa3-9975-6451d29ae1a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.023257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c47959-efe2-4aa3-9975-6451d29ae1a6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.259865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1105/2229] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:25:06 2022] POST /identity/v3/projects => generated 383 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:06.264774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.283889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee206961-2b3f-4b82-94f4-a3cf7ab6ee8a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.392776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.393853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.396427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.396837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.397538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.399578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.414700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596d09-376a-44a8-a314-ca643b13490a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.417132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596d09-376a-44a8-a314-ca643b13490a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.417132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596d09-376a-44a8-a314-ca643b13490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.438884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-40596d09-376a-44a8-a314-ca643b13490a None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:06.440265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-40596d09-376a-44a8-a314-ca643b13490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:06.509397 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1106/2230] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:25:06.535574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596d09-376a-44a8-a314-ca643b13490a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.537075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-40596d09-376a-44a8-a314-ca643b13490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:06.563326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1125/2231] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:25:06 2022] POST /identity/v3/users => generated 292 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:25:06.564438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.576066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af89a249-25bb-460e-a3f6-a08542d0996a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.583789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.584558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.584912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.585249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.585998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.588261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.601350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1126/2232] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:25:06.606227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.612354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596d09-376a-44a8-a314-ca643b13490a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.618093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.618713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.618993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.619261 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.620560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ea78e-c27d-4ee4-9445-27386f312017 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.628217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.628753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b037a889-8a58-4f85-abff-ea07b41ed52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.629380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037a889-8a58-4f85-abff-ea07b41ed52c admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.629660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037a889-8a58-4f85-abff-ea07b41ed52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.629770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.629929 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b037a889-8a58-4f85-abff-ea07b41ed52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.630812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037a889-8a58-4f85-abff-ea07b41ed52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.632591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b037a889-8a58-4f85-abff-ea07b41ed52c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.638682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1107/2233] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:25:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:25:06.647827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1127/2234] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:25:06.652695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.663221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14521509-fd52-401c-81e5-da633ac91470 service swift] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:06.668443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:06.669090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.669408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.669709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] PATH_INFO: `/v3/projects/049e0892819c4a07b79c39737548839d/users/687fc77f37c54cd5a4503723ae59a082/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.685273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=049e0892819c4a07b79c39737548839d, user_id=687fc77f37c54cd5a4503723ae59a082, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:06.687895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:06.801679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1108/2235] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:25:06 2022] PUT /identity/v3/projects/049e0892819c4a07b79c39737548839d/users/687fc77f37c54cd5a4503723ae59a082/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:25:06.808082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4dd1e-25fc-4c85-88b5-116555cc8879 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:06.808591 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4dd1e-25fc-4c85-88b5-116555cc8879 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:06.809009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b4dd1e-25fc-4c85-88b5-116555cc8879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:06.831299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-07b4dd1e-25fc-4c85-88b5-116555cc8879 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:06.832959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-07b4dd1e-25fc-4c85-88b5-116555cc8879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:06.935094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1128/2236] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:06.945438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:06.947378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:07.062874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869ac22b-90bc-4b83-a4bc-3bcd21a0d387 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:07.069527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:07.082376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:07.082376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:07.082376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:07.085422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:07.087134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:07.099693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1109/2237] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:25:07.658519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:08.217826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:09.377025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf275ee-f4dc-44ac-b2d7-867d68c25c56 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:09.379340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf275ee-f4dc-44ac-b2d7-867d68c25c56 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:09.380112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf275ee-f4dc-44ac-b2d7-867d68c25c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:09.448182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0bf275ee-f4dc-44ac-b2d7-867d68c25c56 None None] MFA Rules not processed for user `687fc77f37c54cd5a4503723ae59a082`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:09.449877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0bf275ee-f4dc-44ac-b2d7-867d68c25c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:09.621070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1129/2238] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:09.645865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:09.648189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:09.737990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a53eb-a6f7-43a9-bae1-fccb866ffb84 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:09.744565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb414f95-cc43-4188-9b87-9f98a8c59259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:09.745027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb414f95-cc43-4188-9b87-9f98a8c59259 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:09.745445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb414f95-cc43-4188-9b87-9f98a8c59259 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:09.746063 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb414f95-cc43-4188-9b87-9f98a8c59259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:09.763075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb414f95-cc43-4188-9b87-9f98a8c59259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:09.764626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb414f95-cc43-4188-9b87-9f98a8c59259 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:09.777781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1110/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:11.925303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:11.940857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bccc27-d7f0-4e6f-9d3c-9d67156dfb10 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:11.941585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bccc27-d7f0-4e6f-9d3c-9d67156dfb10 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:11.942082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bccc27-d7f0-4e6f-9d3c-9d67156dfb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:11.967414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b5bccc27-d7f0-4e6f-9d3c-9d67156dfb10 None None] MFA Rules not processed for user `9b5890ee9a114299b0a15a6608503632`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:11.969037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b5bccc27-d7f0-4e6f-9d3c-9d67156dfb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:12.052102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1130/2240] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:12.669663 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:13.598949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6ae6-e4af-4b85-81a3-bf48b3250044 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:13.598949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6ae6-e4af-4b85-81a3-bf48b3250044 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:13.599589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d6ae6-e4af-4b85-81a3-bf48b3250044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:13.626367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-7a5d6ae6-e4af-4b85-81a3-bf48b3250044 None None] MFA Rules not processed for user `687fc77f37c54cd5a4503723ae59a082`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:13.627893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7a5d6ae6-e4af-4b85-81a3-bf48b3250044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:13.635558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1111/2241] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:14.168860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:17.025729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:21.071606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:21.080040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:21.080040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:21.092961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] MFA Rules not processed for user `e49a2b5153b04661b3ef6fe699867005`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:21.094172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:21.149907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1131/2242] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:25:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:21.793807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:23.662336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:27.626750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:27.627250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:27.627250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:27.659673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:25:27.661480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:27.743886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:27.773931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1112/2243] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:25:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:25:27.784660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:27.881491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e31f3-3391-401e-9d3c-8c0c16d204f2 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:27.887706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:27.888424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:27.888769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:27.889130 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:27.898464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:27.900266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:27.910758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1132/2244] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:25:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:25:37.907389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=82 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:39.513446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:25:44.811735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:25:44.813323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:44.891353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014be69a-a792-4770-a472-1e361bd07a88 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:25:44.896439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:25:44.897067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:25:44.897349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:25:44.897626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:25:44.899452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:25:44.913393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:25:44.915019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac1126-fec3-41a6-a580-612cceeff208 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:25:44.926597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1113/2245] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:25:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:25:54.926702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=32 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:17.118096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:17.120105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:17.134677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541fdd8f-a4f5-4d1f-9723-cea37f50ce9c service swift] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:17.139812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:17.140526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:17.140806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:17.141075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] PATH_INFO: `/v3/users/eea9707b4ca04ed190b6d5eca313a93d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:17.142189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea9707b4ca04ed190b6d5eca313a93d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:17.144145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:17.390781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea9707b4ca04ed190b6d5eca313a93d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:17.391596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea9707b4ca04ed190b6d5eca313a93d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:17.392302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea9707b4ca04ed190b6d5eca313a93d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:17.399343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea9707b4ca04ed190b6d5eca313a93d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:17.478513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea9707b4ca04ed190b6d5eca313a93d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:17.502700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1133/2246] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:26:17 2022] DELETE /identity/v3/users/eea9707b4ca04ed190b6d5eca313a93d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:26:17.507519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:17.508023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:17.508259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:17.528638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:17.529872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:17.593011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1114/2247] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:17.602393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:17.608754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:17.670734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fddebc-2c45-4204-a857-a99e41b69c53 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:17.683414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa5a85-70e8-43e1-8d60-8f50c9571aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:17.684066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5a85-70e8-43e1-8d60-8f50c9571aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:17.684397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5a85-70e8-43e1-8d60-8f50c9571aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:17.684712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa5a85-70e8-43e1-8d60-8f50c9571aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:17.695783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa5a85-70e8-43e1-8d60-8f50c9571aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:17.697254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa5a85-70e8-43e1-8d60-8f50c9571aa5 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:17.708215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1134/2248] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:18.204132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:18.206152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:18.220100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa0fa0-24b2-477e-ad1d-b931a20a0184 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:18.224786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:18.225448 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:18.225733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:18.226007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] PATH_INFO: `/v3/projects/ba121949bf75417ebaf0aaf183f897bb` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:18.227189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba121949bf75417ebaf0aaf183f897bb)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:18.231382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:18.404613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba121949bf75417ebaf0aaf183f897bb', 'request_id': 'req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:18.413405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba121949bf75417ebaf0aaf183f897bb', 'request_id': 'req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:18.420972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba121949bf75417ebaf0aaf183f897bb', 'request_id': 'req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:18.535971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] The token cache is being invalidate because project ba121949bf75417ebaf0aaf183f897bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:26:18.536646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:18.536968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-daea3640-f4d4-4c73-a3d3-1a90ca765dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:26:18.538465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1115/2249] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:26:18 2022] DELETE /identity/v3/projects/ba121949bf75417ebaf0aaf183f897bb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:26:18.544868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:18.545480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:18.545480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:18.565779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:18.567118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:18.630228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1135/2250] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:18.638880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:18.639168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:18.639445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:18.663313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:18.664529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:18.672871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1116/2251] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:18.677928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:18.690640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1f8c3-b6d0-4a56-addf-c1f5c3ff3934 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:18.695762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:18.696433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:18.697024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:18.697311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:18.698448 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:26:18.700013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:18.700711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:18.717974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1136/2252] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:26:18.733155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:18.743762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6417bf2-d5dd-4540-9faa-ce928b460719 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:18.749180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:18.749911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:18.750502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:18.750996 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:18.751729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:18.753660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:18.909987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1117/2253] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:26:18 2022] POST /identity/v3/projects => generated 389 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:26:18.915126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:18.920686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:18.926106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63753b-ab4d-446b-a07c-24d727d451eb admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:18.990536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:18.991173 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:18.991542 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:18.991808 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:18.992482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:18.995047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:19.145267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1137/2254] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:26:18 2022] POST /identity/v3/users => generated 295 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:26:19.148905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:19.159558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0aed-929b-4fa9-bdba-153f1a6a8f62 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:19.164308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:19.164909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:19.165235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:19.165455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:19.166123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:19.168038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:19.179595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1118/2255] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:26:19.191751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:19.204155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031039-3529-4465-adc5-39d9b3fd3749 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:19.209295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f80f52-d861-4563-9c1f-744da0a7b4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:19.210085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80f52-d861-4563-9c1f-744da0a7b4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:19.210364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80f52-d861-4563-9c1f-744da0a7b4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:19.210650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f80f52-d861-4563-9c1f-744da0a7b4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:19.211387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80f52-d861-4563-9c1f-744da0a7b4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:19.213051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f80f52-d861-4563-9c1f-744da0a7b4c3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:19.228163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1138/2256] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:26:19.232743 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:19.245928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86228-7ca5-4648-83d3-b2038ada6ffa admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:19.250623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:19.251305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:19.252260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:19.252260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] PATH_INFO: `/v3/projects/d0cdfad0404a4e30835a64a1314f4d48/users/4aab4e3926254a8aa6c39ca41911eff1/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:19.270649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0cdfad0404a4e30835a64a1314f4d48, user_id=4aab4e3926254a8aa6c39ca41911eff1, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:19.273522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:19.411023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1119/2257] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:26:19 2022] PUT /identity/v3/projects/d0cdfad0404a4e30835a64a1314f4d48/users/4aab4e3926254a8aa6c39ca41911eff1/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:26:19.422405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:19.422694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:19.422973 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:19.461536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] MFA Rules not processed for user `4aab4e3926254a8aa6c39ca41911eff1`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:19.462971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:19.542329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1139/2258] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:26:19 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:19.553515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:19.555391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:19.577986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:19.614710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597821c8-2131-4227-8279-e5b05bc3c6fa admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:19.618799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92393583-b713-4b0e-9cc0-e2406c0d95f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:19.619489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92393583-b713-4b0e-9cc0-e2406c0d95f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:19.620673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92393583-b713-4b0e-9cc0-e2406c0d95f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:19.620673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92393583-b713-4b0e-9cc0-e2406c0d95f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:19.632072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92393583-b713-4b0e-9cc0-e2406c0d95f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:19.633521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92393583-b713-4b0e-9cc0-e2406c0d95f4 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:19.643974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1120/2259] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:26:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:25.608763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:25.610172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:25.665295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76388ae6-ba98-4d19-a9eb-99a840210891 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:25.669504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:25.670111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:25.670389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:25.670659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:25.672385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:25.698080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:25.736946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:25.738369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:25.748434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1140/2260] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:26:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:26:29.809634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=17 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:35.965602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=18 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:26:52.029555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27368c5f-9820-44e6-8e8b-dc91aba1fe8f None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:52.032799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27368c5f-9820-44e6-8e8b-dc91aba1fe8f None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:52.032799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27368c5f-9820-44e6-8e8b-dc91aba1fe8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:52.050017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-27368c5f-9820-44e6-8e8b-dc91aba1fe8f None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:52.051228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-27368c5f-9820-44e6-8e8b-dc91aba1fe8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:52.121889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1121/2261] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:26:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:26:52.181729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:26:52.181729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:52.238421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dac6c2-707e-4f21-9739-371a445fa8e8 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:26:52.244529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:26:52.244529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:52.244529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:52.244854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:52.253935 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:26:52.255533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:26:52.263546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1141/2262] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:26:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:26:54.587195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe084e08-74ce-453f-a623-e759319108cd None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:26:54.587195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe084e08-74ce-453f-a623-e759319108cd None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:26:54.587195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe084e08-74ce-453f-a623-e759319108cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:26:54.592671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-fe084e08-74ce-453f-a623-e759319108cd None None] MFA Rules not processed for user `0782e27e2318438fbf155dfa9cb834b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:26:54.593997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fe084e08-74ce-453f-a623-e759319108cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:26:54.660953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1122/2263] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:26:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:00.892300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:00.968781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c59352-06e0-469e-b4f6-0d8f513fb85a service swift] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:00.974839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:00.975518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:00.975793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:00.976146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] PATH_INFO: `/v3/users/4aab4e3926254a8aa6c39ca41911eff1` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:00.977433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aab4e3926254a8aa6c39ca41911eff1)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:00.984090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:02.247137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aab4e3926254a8aa6c39ca41911eff1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:02.247881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aab4e3926254a8aa6c39ca41911eff1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:02.248537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aab4e3926254a8aa6c39ca41911eff1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:02.256085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aab4e3926254a8aa6c39ca41911eff1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:02.662085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aab4e3926254a8aa6c39ca41911eff1'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:02.686045 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1142/2264] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:27:00 2022] DELETE /identity/v3/users/4aab4e3926254a8aa6c39ca41911eff1 => generated 0 bytes in 1794 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:27:02.690547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:02.691589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:02.692062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:02.713476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:02.714934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:02.833620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1123/2265] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:02.847853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:02.851042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:02.866080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:02.928417 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ea8dfd-1f8d-4df8-a5f6-d7f6ab69959e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:02.930858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:02.931482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:02.931923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:02.933001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:02.943795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:02.945273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:02.971274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1143/2266] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:04.165138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:04.168889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:04.170299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:04.190990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af43a7-c053-4f6c-8452-388bb69ff5d2 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:04.194048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f796ff-7d0e-48ea-89e8-8d2bf770fdda None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:04.200439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:04.203849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:04.206818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:04.207328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:04.207452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:04.207727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:04.208011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] PATH_INFO: `/v3/projects/d0cdfad0404a4e30835a64a1314f4d48` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:04.208353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:04.209337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0cdfad0404a4e30835a64a1314f4d48)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:04.211240 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:04.211880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:04.326122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:04.349959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:04.351514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3539e-0aa5-434c-b55b-6ec7730445f5 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:04.376435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1124/2267] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:04.519715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0cdfad0404a4e30835a64a1314f4d48', 'request_id': 'req-0a8d95fa-988f-4e4a-adfd-56df08694f41'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:04.528714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0cdfad0404a4e30835a64a1314f4d48', 'request_id': 'req-0a8d95fa-988f-4e4a-adfd-56df08694f41'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:04.536078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0cdfad0404a4e30835a64a1314f4d48', 'request_id': 'req-0a8d95fa-988f-4e4a-adfd-56df08694f41'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:04.767703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] The token cache is being invalidate because project d0cdfad0404a4e30835a64a1314f4d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:27:04.769547 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:04.772453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-0a8d95fa-988f-4e4a-adfd-56df08694f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:27:04.774858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1144/2268] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:27:04 2022] DELETE /identity/v3/projects/d0cdfad0404a4e30835a64a1314f4d48 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:27:14.965329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=49,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:15.073281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:25.167013 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75c2ed-7d05-4500-b9d0-0690b77c876c None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:25.167633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75c2ed-7d05-4500-b9d0-0690b77c876c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:25.167633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75c2ed-7d05-4500-b9d0-0690b77c876c None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:25.168892 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1125/2269] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:27:25.174908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b44db-1fbf-4f8d-a495-cf757e016624 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:25.177606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b44db-1fbf-4f8d-a495-cf757e016624 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:25.177606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b44db-1fbf-4f8d-a495-cf757e016624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:25.195010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a60b44db-1fbf-4f8d-a495-cf757e016624 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:25.196258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a60b44db-1fbf-4f8d-a495-cf757e016624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:25.262229 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1145/2270] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:27:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:25.273430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75c2ed-7d05-4500-b9d0-0690b77c876c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:25.275629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-df75c2ed-7d05-4500-b9d0-0690b77c876c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:25.332213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75c2ed-7d05-4500-b9d0-0690b77c876c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:25.335752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984136f-3c73-4167-9263-c3e465d81408 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:25.336370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984136f-3c73-4167-9263-c3e465d81408 service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:25.336652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984136f-3c73-4167-9263-c3e465d81408 service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:25.336921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984136f-3c73-4167-9263-c3e465d81408 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:25.347144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984136f-3c73-4167-9263-c3e465d81408 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:25.348584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984136f-3c73-4167-9263-c3e465d81408 service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:25.359371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1126/2271] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:27:25.879974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4e15e-7a61-4600-b220-5cc1cec7a6f8 None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:25.880353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4e15e-7a61-4600-b220-5cc1cec7a6f8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:25.880631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f4e15e-7a61-4600-b220-5cc1cec7a6f8 None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:25.882024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1146/2272] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:27:25.889909 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d7ac4-9aa1-4f03-8eee-2e4a6535ab3d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:25.890190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d7ac4-9aa1-4f03-8eee-2e4a6535ab3d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:25.890463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d7ac4-9aa1-4f03-8eee-2e4a6535ab3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:25.910374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-bb7d7ac4-9aa1-4f03-8eee-2e4a6535ab3d None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:25.911782 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bb7d7ac4-9aa1-4f03-8eee-2e4a6535ab3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:25.968983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1127/2273] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:27:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:25.981461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4e15e-7a61-4600-b220-5cc1cec7a6f8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:25.983204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-87f4e15e-7a61-4600-b220-5cc1cec7a6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:25.997433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f4e15e-7a61-4600-b220-5cc1cec7a6f8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:26.008067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2109734-a252-4281-8926-6d43542031da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:26.008067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2109734-a252-4281-8926-6d43542031da service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:26.008067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2109734-a252-4281-8926-6d43542031da service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:26.008067 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2109734-a252-4281-8926-6d43542031da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:26.010352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2109734-a252-4281-8926-6d43542031da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:26.011933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2109734-a252-4281-8926-6d43542031da service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:26.020137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1147/2274] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:26.102194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708fd0-ab06-4f39-afcb-936b73235457 None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:26.102479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708fd0-ab06-4f39-afcb-936b73235457 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:26.102760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708fd0-ab06-4f39-afcb-936b73235457 None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:26.104454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1128/2275] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:27:26.109664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:26.109942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:26.110213 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:26.129788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:26.131024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:26.140810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1148/2276] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:27:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:26.151232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9708fd0-ab06-4f39-afcb-936b73235457 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:26.152962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f9708fd0-ab06-4f39-afcb-936b73235457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:26.166456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9708fd0-ab06-4f39-afcb-936b73235457 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:26.170125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:26.170716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:26.170993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:26.171264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:26.184484 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:26.185916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:26.198266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1129/2277] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:27.060681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:27.070243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6998d1f-c1ff-4d0b-ba1a-7db12e05ad61 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:27.074352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:27.074943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:27.075219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:27.075487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:27.077222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:27.092795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:27.094499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f7fae9-57f7-41f8-8a70-e94ab0251a86 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:27.105766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1149/2278] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:27.426968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:27.436271 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9107a-1ccb-4e69-b598-ca3f02cb6768 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:27.439862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:27.440479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:27.440757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:27.441027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:27.442736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:27.456469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:27.457874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:27.468813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1130/2279] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:37.100774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=46 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:37.463804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=61 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:40.105611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:40.106110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:40.106220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:40.129543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:27:40.131094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:40.189846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1150/2280] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:27:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:27:40.203177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:40.204968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:40.276948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2aea50-4094-4241-8ac4-9e9b0f0f64b0 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:40.284752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:40.284752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:40.284752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:40.284752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:40.292113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:40.293687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:40.302097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1131/2281] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:27:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:27:44.033490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:27:44.034969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:44.050952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929eed8-9e5f-4da1-8625-eeb2009283a6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:27:44.056730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:27:44.057062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:27:44.057336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:27:44.057604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:27:44.059254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:27:44.118153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:27:44.119768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:27:44.128766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1151/2282] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:27:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:27:50.298441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=32 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:27:54.124539 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=53 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:28:07.900199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:07.912405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b4184c-98a6-439f-a3d1-d867be52e258 service swift] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:07.918918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:07.919515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:07.919797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:07.920095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:07.921793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:07.934959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:07.936616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc49f7-2c58-4480-bc66-4dbc7cfc5ae8 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:07.946705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1132/2283] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:28:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:14.669422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:14.678582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf9de4-d369-4d41-9023-0086429f4796 service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:14.682951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:14.683541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:14.683816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:14.684120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:14.685794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:14.694915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:14.696352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:14.703940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1152/2284] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:28:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:17.942649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:28:18.026843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602db7e4-7e8e-4eb7-a5b3-9da34756256b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:18.027128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602db7e4-7e8e-4eb7-a5b3-9da34756256b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:18.027407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602db7e4-7e8e-4eb7-a5b3-9da34756256b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:18.048324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-602db7e4-7e8e-4eb7-a5b3-9da34756256b None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:28:18.049553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-602db7e4-7e8e-4eb7-a5b3-9da34756256b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:18.056995 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1133/2285] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 10 11:28:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:28:18.080926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:18.093907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359697-e30f-458f-b39a-7bdbd6ab3148 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:18.098860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:18.099461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:18.099736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:18.100057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:18.109381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:18.110984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:18.120471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1153/2286] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:28:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:28:23.348785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:23.349338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:23.349338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:23.368124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] MFA Rules not processed for user `687fc77f37c54cd5a4503723ae59a082`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:28:23.369870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:23.428164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1134/2287] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:28:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:28:23.689365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:23.698829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f2ef8-5165-4581-aba9-8359a3a8f887 service swift] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:23.702309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:23.702890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:23.703164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:23.703430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:23.705431 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:23.714998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:23.716433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:23.724190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1154/2288] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:28:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:24.164510 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:24.165802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:24.238089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd1ce5-55cf-4855-bfc2-08b5837e027a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:24.242922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:24.243517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:24.243790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:24.244130 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:24.245863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:24.256852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:24.258406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:24.267985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1135/2289] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:28:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:33.720746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=18 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:28:34.263831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=60 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:28:36.893442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:28:36.908759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f6c5d-efa1-4780-beca-1faf2a70d22b service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:28:36.914340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:28:36.914960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:28:36.915236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:28:36.915499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:28:36.917716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:28:36.930615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:28:36.937301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:28:36.944505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1155/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:28:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:28:46.939344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=6 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:29:04.170091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:04.171457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:04.186459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c82b124-ed32-4843-8120-02dfe23083e6 service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:04.191819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:04.197249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:04.197249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:04.197249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:04.197249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:04.252717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:04.254247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:04.265546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1136/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:29:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:29:04.268376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:04.283046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c2ec4-1ccc-4325-b8ad-557de7c15506 service nova] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:04.288612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:04.289251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:04.289523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:04.289793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:04.291728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:04.353726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:04.357334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4866cb5-974e-4675-8f1c-50632f57f11d service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:04.366688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1156/2292] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:29:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:29:14.261244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=28 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:29:14.362231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=28 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:29:20.847855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:20.860786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44375-beeb-40c6-b0f9-a47aaceef0ab service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:20.867641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:20.868904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:20.869230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:20.869599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:20.872228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:20.886833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:20.888514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:20.899215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1137/2293] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:29:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:29:30.894772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:29:47.648251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:47.648818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:47.648818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:47.669134 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] MFA Rules not processed for user `9b5890ee9a114299b0a15a6608503632`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:29:47.670338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:47.677263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1157/2294] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:29:47 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:29:57.663129 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:29:58.221491 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:58.233149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba24d08-bdb9-4ef7-92a5-136a705a34ff service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:58.238834 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:58.239514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:58.239795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:58.240112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:58.241833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:58.300112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:58.301713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:58.311144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1138/2295] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:29:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:29:58.696565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:58.698071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:58.712951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0d2e-aa43-4fb3-89d7-65f8a7de9366 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:58.717890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:58.718498 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:58.718768 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:58.719036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:58.720752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:58.732663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:58.736807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:58.744609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1158/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:29:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:29:58.831010 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:58.832726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:58.897791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5df84e-0e97-422c-bfc1-2c778f1fa415 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:58.902907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:58.903565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:58.903841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:58.904145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] PATH_INFO: `/v3/users/687fc77f37c54cd5a4503723ae59a082` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:58.905305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=687fc77f37c54cd5a4503723ae59a082)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:58.907216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:59.122715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '687fc77f37c54cd5a4503723ae59a082'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.123366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '687fc77f37c54cd5a4503723ae59a082'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.123974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '687fc77f37c54cd5a4503723ae59a082'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.130723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '687fc77f37c54cd5a4503723ae59a082'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.226402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-f22dec87-e645-4bcc-8b5d-0ad726c32488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '687fc77f37c54cd5a4503723ae59a082'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.243593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1139/2297] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:29:58 2022] DELETE /identity/v3/users/687fc77f37c54cd5a4503723ae59a082 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:29:59.535850 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:29:59.551116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9663a74d-225b-4ccb-9675-2c54bb902a3a service placement] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:29:59.623168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:29:59.628091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:59.628091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:59.628091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] PATH_INFO: `/v3/projects/049e0892819c4a07b79c39737548839d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:59.628091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=049e0892819c4a07b79c39737548839d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:29:59.628091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:29:59.811210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '049e0892819c4a07b79c39737548839d', 'request_id': 'req-1048d15f-825d-4f81-9f31-984c6b775cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.817205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '049e0892819c4a07b79c39737548839d', 'request_id': 'req-1048d15f-825d-4f81-9f31-984c6b775cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.823682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '049e0892819c4a07b79c39737548839d', 'request_id': 'req-1048d15f-825d-4f81-9f31-984c6b775cd4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.923710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] The token cache is being invalidate because project 049e0892819c4a07b79c39737548839d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:29:59.924137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.924828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1048d15f-825d-4f81-9f31-984c6b775cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:29:59.926363 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1159/2298] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:29:59 2022] DELETE /identity/v3/projects/049e0892819c4a07b79c39737548839d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:29:59.931308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:29:59.931593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:29:59.931929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:29:59.951900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:29:59.953143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:29:59.966244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:00.017585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1140/2299] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:00.025927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.026207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.026480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.046151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:00.047262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:00.055364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1160/2300] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:30:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:00.059795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.069309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a15c68-c055-4480-b962-4215deb96c91 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.074828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.075416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.075692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.075960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.076633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:30:00.076932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.078505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.090795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1141/2301] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:30:00.095021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.104560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c35bb-d295-43e5-8a89-75e5fd17ed90 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.111348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.111348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.111348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.111348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.112333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.112973 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.233571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1161/2302] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:30:00 2022] POST /identity/v3/projects => generated 399 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:30:00.237722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.247796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815590c-78db-48d2-a2ad-9682c9cca40c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.302630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.303233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.303511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.303781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.304433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.306524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.414751 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1142/2303] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:30:00 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:30:00.418987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.429403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbf024-7e10-485a-9d24-594e42125e31 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.434821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.435422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.435700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.436117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.436759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.438377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.449005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1162/2304] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:30:00.452963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.463460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f135c-0ec4-449d-8422-3bad782816f2 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.467974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7643c32-e174-4fc4-8360-dccbe897997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.468599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643c32-e174-4fc4-8360-dccbe897997e admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.483685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643c32-e174-4fc4-8360-dccbe897997e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.483685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7643c32-e174-4fc4-8360-dccbe897997e admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.483685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643c32-e174-4fc4-8360-dccbe897997e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.483685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7643c32-e174-4fc4-8360-dccbe897997e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.483685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1143/2305] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:30:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:30:00.486459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.497000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca22ac54-b363-4eaa-85f5-42e82810b899 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.501683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.502347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.502627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.502903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] PATH_INFO: `/v3/projects/253ba7559fb245b7ac962ae44bd56683/users/7a3f371847ce420ab44bd3833b7cc6fd/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.517920 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=253ba7559fb245b7ac962ae44bd56683, user_id=7a3f371847ce420ab44bd3833b7cc6fd, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.520697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.656060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1163/2306] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:30:00 2022] PUT /identity/v3/projects/253ba7559fb245b7ac962ae44bd56683/users/7a3f371847ce420ab44bd3833b7cc6fd/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:30:00.666696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.666974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.667246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.695552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] MFA Rules not processed for user `7a3f371847ce420ab44bd3833b7cc6fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:00.697522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:00.764789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1144/2307] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:30:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:00.776998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:00.779590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:00.794080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:00.838853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bfd37-4191-4211-bfcc-5a2bdff7d9e6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:00.841865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:00.842478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:00.842764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:00.843035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:00.851521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:00.853091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:00.862446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1164/2308] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:06.528707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:06.530102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:06.544736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:06.604048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b970bb-a429-43f8-9f51-055519a3c313 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:06.609715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:06.610328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:06.610641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:06.610915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:06.612633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:06.673328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:06.674945 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f47909c-70f2-4233-848f-e35679d7c324 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:06.684080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1145/2309] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:10.969142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=20 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:12.320472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:12.398242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265547c9-7362-4cf8-be91-0054d4421d47 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:12.403324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:12.403985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:12.404289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:12.404559 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] PATH_INFO: `/v3/users/7a3f371847ce420ab44bd3833b7cc6fd` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:12.405683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3f371847ce420ab44bd3833b7cc6fd)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:12.407597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:12.841335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3f371847ce420ab44bd3833b7cc6fd'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:12.842570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3f371847ce420ab44bd3833b7cc6fd'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:12.842766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3f371847ce420ab44bd3833b7cc6fd'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:12.861763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3f371847ce420ab44bd3833b7cc6fd'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:13.015626 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3f371847ce420ab44bd3833b7cc6fd'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:13.043172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1165/2310] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:30:12 2022] DELETE /identity/v3/users/7a3f371847ce420ab44bd3833b7cc6fd => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:30:13.047791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:13.048353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:13.048832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:13.068297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:13.069750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:13.140163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1146/2311] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:13.149711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:13.151379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:13.211629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b5692-4efb-4cca-b935-7b42835cda82 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:13.216021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547c6d1-ec28-4c36-958e-398a7d66606c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:13.216718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547c6d1-ec28-4c36-958e-398a7d66606c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:13.216993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547c6d1-ec28-4c36-958e-398a7d66606c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:13.217258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547c6d1-ec28-4c36-958e-398a7d66606c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:13.226776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547c6d1-ec28-4c36-958e-398a7d66606c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:13.228403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547c6d1-ec28-4c36-958e-398a7d66606c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:13.239289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1166/2312] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:13.885953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:13.885953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:13.899529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a84c3ba-3d1b-4ae4-bd98-8e5bcecc2214 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:13.906796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:13.907522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:13.907880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:13.908928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] PATH_INFO: `/v3/projects/253ba7559fb245b7ac962ae44bd56683` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:13.911646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=253ba7559fb245b7ac962ae44bd56683)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:13.913972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:14.077926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:14.081521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '253ba7559fb245b7ac962ae44bd56683', 'request_id': 'req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:14.089561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '253ba7559fb245b7ac962ae44bd56683', 'request_id': 'req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:14.097040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '253ba7559fb245b7ac962ae44bd56683', 'request_id': 'req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:14.207305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] The token cache is being invalidate because project 253ba7559fb245b7ac962ae44bd56683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:30:14.207598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:14.208284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:30:14.219835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1147/2313] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:30:13 2022] DELETE /identity/v3/projects/253ba7559fb245b7ac962ae44bd56683 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:30:23.235150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:23.727278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1b61-2693-4369-ad50-9ef7be173cd1 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:23.727461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1b61-2693-4369-ad50-9ef7be173cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:23.727961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc1b61-2693-4369-ad50-9ef7be173cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:23.769308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-7ffc1b61-2693-4369-ad50-9ef7be173cd1 None None] MFA Rules not processed for user `87adb637dd814cf8be9ff6c2a2bc26ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:23.770959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7ffc1b61-2693-4369-ad50-9ef7be173cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:23.966335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1167/2314] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:30:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:24.235385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:24.279653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:24.281089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:24.345615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79bfb4-f7c3-47cc-a274-81320c69ac1a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:24.349212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:24.349819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:24.350128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:24.350369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:24.352145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:24.363001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:24.364493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:24.374371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1148/2315] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:31.400606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:31.401141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:31.401141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:31.429209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] MFA Rules not processed for user `0782e27e2318438fbf155dfa9cb834b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:31.434177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:31.506990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1168/2316] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:30:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:34.386678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=26 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:41.504949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=62 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:43.218183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:43.219629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:43.365622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327f255-a965-45e7-98e6-ca5b858cfb0e service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:43.373618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:43.373884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:43.374164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:43.374436 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:43.376193 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:43.433428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:43.434921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c89feb-e88d-4057-99a8-1b945b851745 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:43.443907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1149/2317] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:50.627650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:50.629893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:50.756203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ed140-1b66-4e39-bced-5b4601995c02 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:50.756203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:50.756203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:50.756203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:50.756694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:50.759292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:50.772282 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:50.773984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:50.785704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1169/2318] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:53.440015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=84 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:30:56.803128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:56.804598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:56.804598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:56.841915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:56.843195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:56.905443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1150/2319] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 10 11:30:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:30:57.248503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:57.249888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:57.332948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a00d-8fd3-494b-bc43-28d1f8aaf841 service placement] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:57.338402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aead93b-02f5-4730-a8db-f0a71add0601 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:57.344263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead93b-02f5-4730-a8db-f0a71add0601 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:57.344263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead93b-02f5-4730-a8db-f0a71add0601 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:57.344263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aead93b-02f5-4730-a8db-f0a71add0601 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:57.349199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead93b-02f5-4730-a8db-f0a71add0601 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:57.351013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aead93b-02f5-4730-a8db-f0a71add0601 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:57.361288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1170/2320] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:30:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:30:59.062830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:30:59.090554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677db6fc-a687-4e88-afde-cfe3db999243 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:30:59.090554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:30:59.090554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:59.090554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:59.091459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:59.093209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:59.190152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:30:59.191520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:30:59.211871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1151/2321] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:30:59.650674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:30:59.651958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:30:59.651958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:30:59.671620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] MFA Rules not processed for user `11be37af210c45fc9ddebf7f28b265ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:30:59.672941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:30:59.730937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1171/2322] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:30:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:31:03.850783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:03.863173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea529323-8bda-4aaa-a6ee-0db802978dfa service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:03.868401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:03.869005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:03.869282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:03.869557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:03.872313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:03.883608 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:03.885314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:03.896200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1152/2323] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:31:08.894544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:08.896068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:08.905922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:08.910396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94a8d0-5108-4c53-9b6e-314d1ce48554 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:08.914880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:08.915507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:08.915782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:08.916089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] PATH_INFO: `/v3/users/e49a2b5153b04661b3ef6fe699867005` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:08.917400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49a2b5153b04661b3ef6fe699867005)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:08.919372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:09.174851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49a2b5153b04661b3ef6fe699867005'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.175641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49a2b5153b04661b3ef6fe699867005'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.176194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49a2b5153b04661b3ef6fe699867005'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.191096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49a2b5153b04661b3ef6fe699867005'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.286916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49a2b5153b04661b3ef6fe699867005'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.310378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1172/2324] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:31:08 2022] DELETE /identity/v3/users/e49a2b5153b04661b3ef6fe699867005 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:09.579861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:09.603728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0937f-fcbb-44ec-89c0-f02097bde3e8 service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:09.645767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:09.663758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:09.664442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:09.664747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:09.665023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] PATH_INFO: `/v3/projects/e202fe4250104681a9fe73fc4750c33f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:09.666179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e202fe4250104681a9fe73fc4750c33f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:09.668794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:09.861162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e202fe4250104681a9fe73fc4750c33f', 'request_id': 'req-ee516ded-af37-4dcb-80c8-f11bb9ede593'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.867303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e202fe4250104681a9fe73fc4750c33f', 'request_id': 'req-ee516ded-af37-4dcb-80c8-f11bb9ede593'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:09.874174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e202fe4250104681a9fe73fc4750c33f', 'request_id': 'req-ee516ded-af37-4dcb-80c8-f11bb9ede593'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.077674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] The token cache is being invalidate because project e202fe4250104681a9fe73fc4750c33f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:31:10.078270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.078923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.080806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1153/2325] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:31:09 2022] DELETE /identity/v3/projects/e202fe4250104681a9fe73fc4750c33f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:10.086135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:10.088402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:10.102551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4414812-1d63-4fa7-8b69-fc66cc4fc8fe admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:10.162576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:10.163251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:10.163531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:10.163801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] PATH_INFO: `/v3/users/1b378d9208e9456dac3744ebecd4ae97` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:10.164993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b378d9208e9456dac3744ebecd4ae97)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:10.167545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:10.450338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b378d9208e9456dac3744ebecd4ae97'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.451036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b378d9208e9456dac3744ebecd4ae97'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.451657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b378d9208e9456dac3744ebecd4ae97'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.459018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b378d9208e9456dac3744ebecd4ae97'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.610645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b378d9208e9456dac3744ebecd4ae97'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:10.623972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:10.631279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1173/2326] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:31:10 2022] DELETE /identity/v3/users/1b378d9208e9456dac3744ebecd4ae97 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:10.944183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:10.945812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:10.961880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee516ded-af37-4dcb-80c8-f11bb9ede593 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:11.031881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:11.032932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:11.033265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:11.034295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] PATH_INFO: `/v3/projects/380e0d1a6e0745e8b4befee713665c18` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:11.035462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=380e0d1a6e0745e8b4befee713665c18)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:11.037406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:11.168812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '380e0d1a6e0745e8b4befee713665c18', 'request_id': 'req-5e985ecb-8563-424b-95d0-248d412f9f61'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:11.175306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '380e0d1a6e0745e8b4befee713665c18', 'request_id': 'req-5e985ecb-8563-424b-95d0-248d412f9f61'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:11.181356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '380e0d1a6e0745e8b4befee713665c18', 'request_id': 'req-5e985ecb-8563-424b-95d0-248d412f9f61'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:11.315852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:11.320151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] The token cache is being invalidate because project 380e0d1a6e0745e8b4befee713665c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:31:11.320518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:11.321230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:11.323315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1154/2327] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:31:10 2022] DELETE /identity/v3/projects/380e0d1a6e0745e8b4befee713665c18 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:21.420339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:26.228725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:26.230702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:26.299408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c722228d-b677-466b-a429-b9f5d9fc80e7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:26.345706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:26.346296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:26.346570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:26.346838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:26.348565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:26.409722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:26.411137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:26.421047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1174/2328] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:31:30.268054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:30.269549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:30.283576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e985ecb-8563-424b-95d0-248d412f9f61 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:30.288445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:30.289111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:30.289581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:30.289862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] PATH_INFO: `/v3/users/9b5890ee9a114299b0a15a6608503632` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:30.291125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5890ee9a114299b0a15a6608503632)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:30.293878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:30.614184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5890ee9a114299b0a15a6608503632'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:30.614904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5890ee9a114299b0a15a6608503632'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:30.615541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5890ee9a114299b0a15a6608503632'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:30.622815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5890ee9a114299b0a15a6608503632'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:30.719581 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b4d7da9f-ff19-4242-b723-df869d58876f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5890ee9a114299b0a15a6608503632'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:30.737658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1155/2329] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:31:30 2022] DELETE /identity/v3/users/9b5890ee9a114299b0a15a6608503632 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:31.023812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:31.034439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4f1c-88aa-47a9-a337-6c7240e81dbf service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:31.084875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:31.092139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:31.092932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:31.093209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:31.093479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] PATH_INFO: `/v3/projects/526e4ff2ae594d54a53e628df8802a92` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:31.094626 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=526e4ff2ae594d54a53e628df8802a92)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:31.096540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:31.222517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '526e4ff2ae594d54a53e628df8802a92', 'request_id': 'req-46dcb21e-5a68-4729-b9a4-7e7260a57343'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:31.227931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '526e4ff2ae594d54a53e628df8802a92', 'request_id': 'req-46dcb21e-5a68-4729-b9a4-7e7260a57343'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:31.234724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '526e4ff2ae594d54a53e628df8802a92', 'request_id': 'req-46dcb21e-5a68-4729-b9a4-7e7260a57343'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:31.368231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] The token cache is being invalidate because project 526e4ff2ae594d54a53e628df8802a92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:31:31.368522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:31.369151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-46dcb21e-5a68-4729-b9a4-7e7260a57343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:31.370652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1175/2330] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:31:31 2022] DELETE /identity/v3/projects/526e4ff2ae594d54a53e628df8802a92 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:31.375584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:31.376067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:31.376189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:31.434877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:31.436159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:31.493605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1156/2331] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:31.501439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:31.501721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:31.501993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:31.521306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:31.522497 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:31.531009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1176/2332] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:31.535117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:31.545358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab4618-a010-4054-be26-1c470f340d04 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:31.549861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:31.550509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:31.550785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:31.551060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:31.551701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:31:31.552023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:31.553615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:31.565470 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1157/2333] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:31:31.569880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:31.579399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b110ce-5b86-4f23-8166-d7a458f4ea66 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:31.584302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:31.584887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:31.585162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:31.585430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:31.586042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:31.587910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:31.777824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1177/2334] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:31:31 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:31.794434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:31.804640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1541a70-4bc0-4596-8b2d-e0467ac16947 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:31.830715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:31.861766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:31.862369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:31.862649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:31.862922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:31.863553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:31.865487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.097324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1158/2335] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:31:31 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:32.101295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.111401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584159e5-7c1e-46ed-81a2-90e784e6643c admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.115752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.123507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.123507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.123507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.123507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.123507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.129574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1178/2336] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:31:32.133623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.144191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd44129c-3199-4a8b-935f-17ee317e21dc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.148740 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754f146-6de7-4cc8-8732-4b4a533dabdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.149330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754f146-6de7-4cc8-8732-4b4a533dabdb admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.149613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754f146-6de7-4cc8-8732-4b4a533dabdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.149884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754f146-6de7-4cc8-8732-4b4a533dabdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.150524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754f146-6de7-4cc8-8732-4b4a533dabdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.152443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754f146-6de7-4cc8-8732-4b4a533dabdb admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.163666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1159/2337] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:31:32.166865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.176770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d6107-d70b-4f67-93de-86733e9bf33a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.181161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.181789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.183006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.183300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] PATH_INFO: `/v3/projects/722ce37029a94c29bffc395b01f14f70/users/af6679b478124f208d957551857799b6/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.197753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=722ce37029a94c29bffc395b01f14f70, user_id=af6679b478124f208d957551857799b6, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.200504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.306015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1179/2338] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:31:32 2022] PUT /identity/v3/projects/722ce37029a94c29bffc395b01f14f70/users/af6679b478124f208d957551857799b6/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:32.318161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.318451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.318734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.346586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] MFA Rules not processed for user `af6679b478124f208d957551857799b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:32.347857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:32.411863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1160/2339] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:31:32.419474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.481857 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06260760-f955-456d-ba97-ba24520513e4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.486414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.487006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.487284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.487550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.488204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.490152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.599056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1180/2340] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:31:32 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:32.603084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.614645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284faab-0170-40f1-a37b-bb85138d8a71 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.670746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.671353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.671632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.671903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.672676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.674573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.774545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1161/2341] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:31:32 2022] POST /identity/v3/users => generated 296 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:32.778918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.789043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef69bb8-4122-400b-8cf0-b1dc836144a3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.794224 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.794813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.795091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.795364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.795987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.797590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.798871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:32.808683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1181/2342] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:31:32.812673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:32.822624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef72c4-1d0b-4407-bd5e-abbe484ef6e3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:32.827173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:32.827825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:32.828137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:32.828411 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] PATH_INFO: `/v3/projects/4b19d752d465420288dae25dff8bd2fa/users/452ddfd5af59489b90105d173a5e721f/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:32.843477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b19d752d465420288dae25dff8bd2fa, user_id=452ddfd5af59489b90105d173a5e721f, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:32.846145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:32.993937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1162/2343] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:31:32 2022] PUT /identity/v3/projects/4b19d752d465420288dae25dff8bd2fa/users/452ddfd5af59489b90105d173a5e721f/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:32.997900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:33.007803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3452d5-4c01-42d2-be2b-b4521a4312f0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:33.061654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d3457-82f9-430f-8bcb-05ccaa21f356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:33.062251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d3457-82f9-430f-8bcb-05ccaa21f356 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:33.062529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d3457-82f9-430f-8bcb-05ccaa21f356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:33.062799 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d3457-82f9-430f-8bcb-05ccaa21f356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:33.063432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d3457-82f9-430f-8bcb-05ccaa21f356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:33.065037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d3457-82f9-430f-8bcb-05ccaa21f356 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:33.075113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1182/2344] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:31:33.079443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:33.085613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:33.091761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97e159-e290-4d94-924e-627d32fa59cf admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:33.096533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:33.097181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:33.097490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:33.097766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] PATH_INFO: `/v3/projects/4b19d752d465420288dae25dff8bd2fa/users/452ddfd5af59489b90105d173a5e721f/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:33.099704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b19d752d465420288dae25dff8bd2fa, user_id=452ddfd5af59489b90105d173a5e721f, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:33.102529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:33.113134 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1163/2345] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:31:33 2022] PUT /identity/v3/projects/4b19d752d465420288dae25dff8bd2fa/users/452ddfd5af59489b90105d173a5e721f/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:33.123472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:33.124589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:33.124589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:33.153729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] MFA Rules not processed for user `452ddfd5af59489b90105d173a5e721f`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:33.154954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:33.220564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1183/2346] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:33.232071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:33.233734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:33.311181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67730c77-ede9-4709-b806-fe987abd0c85 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:33.314964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:33.315596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:33.315882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:33.316216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:33.326220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:33.330650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:33.337093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1164/2347] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:33 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:31:33.659451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:33.669717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758368a-2e45-48aa-a5c6-30b09d0e6cfb None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:33.673265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62d138-3a09-4c8d-b4f4-9ed3bca9c34a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:33.673851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62d138-3a09-4c8d-b4f4-9ed3bca9c34a service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:33.674123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62d138-3a09-4c8d-b4f4-9ed3bca9c34a service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:33.674388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62d138-3a09-4c8d-b4f4-9ed3bca9c34a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:33.731147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62d138-3a09-4c8d-b4f4-9ed3bca9c34a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:33.732603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62d138-3a09-4c8d-b4f4-9ed3bca9c34a service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:33.745198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1184/2348] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:31:43.849060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=39,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:44.011414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:50.068853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:50.130786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae6c364-c860-42a9-a38e-57f55fd6cfdd service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:50.135915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:50.136567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:50.136844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:50.137114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] PATH_INFO: `/v3/users/af6679b478124f208d957551857799b6` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:50.138225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6679b478124f208d957551857799b6)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:50.140146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:50.409747 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6679b478124f208d957551857799b6'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:50.410404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6679b478124f208d957551857799b6'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:50.411004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6679b478124f208d957551857799b6'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:50.418067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6679b478124f208d957551857799b6'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:50.513281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6679b478124f208d957551857799b6'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:50.531692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1165/2349] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:31:50 2022] DELETE /identity/v3/users/af6679b478124f208d957551857799b6 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:50.536318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:50.536792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:50.536908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:50.555024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:50.556470 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:50.613951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1185/2350] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:50.624528 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:50.625878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:50.680941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677ffafe-4e79-4e98-81b8-ab5f44e2dcf4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:50.685143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:50.685737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:50.686019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:50.686287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:50.694994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:50.696473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:50.706676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1166/2351] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:31:51.207090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:51.208910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:51.228447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e8b8e-8bb7-4b96-bd6c-c893e79b922a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:51.234976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:51.235752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:51.236308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:51.236960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] PATH_INFO: `/v3/projects/722ce37029a94c29bffc395b01f14f70` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:51.238474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=722ce37029a94c29bffc395b01f14f70)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:51.240962 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:51.428761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '722ce37029a94c29bffc395b01f14f70', 'request_id': 'req-fc38d1fd-d45a-42dd-9ec5-356c56382a31'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.434575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '722ce37029a94c29bffc395b01f14f70', 'request_id': 'req-fc38d1fd-d45a-42dd-9ec5-356c56382a31'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.440517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '722ce37029a94c29bffc395b01f14f70', 'request_id': 'req-fc38d1fd-d45a-42dd-9ec5-356c56382a31'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.587664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] The token cache is being invalidate because project 722ce37029a94c29bffc395b01f14f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:31:51.591874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.591874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.591874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1186/2352] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:31:51 2022] DELETE /identity/v3/projects/722ce37029a94c29bffc395b01f14f70 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:51.595624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:51.605937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ad892-dac7-4e8e-86cc-9e9292296662 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:51.624590 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:51.662874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:51.663489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:51.663762 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:51.664055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] PATH_INFO: `/v3/users/452ddfd5af59489b90105d173a5e721f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:51.665181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=452ddfd5af59489b90105d173a5e721f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:51.667150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:51.896460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452ddfd5af59489b90105d173a5e721f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.897036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452ddfd5af59489b90105d173a5e721f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.897653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452ddfd5af59489b90105d173a5e721f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:51.906202 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452ddfd5af59489b90105d173a5e721f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:52.002861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-dbea1506-617b-4dd5-9f5f-96d1eaa70d4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452ddfd5af59489b90105d173a5e721f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:52.021455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1167/2353] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:31:51 2022] DELETE /identity/v3/users/452ddfd5af59489b90105d173a5e721f => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:52.548611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:52.550338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:52.566101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d1fd-d45a-42dd-9ec5-356c56382a31 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:52.623988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:52.630181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:52.630181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:52.630181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] PATH_INFO: `/v3/projects/4b19d752d465420288dae25dff8bd2fa` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:52.630181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b19d752d465420288dae25dff8bd2fa)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:52.630181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:52.852344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:52.868311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b19d752d465420288dae25dff8bd2fa', 'request_id': 'req-02bbba61-2f02-4126-a023-9187f4ef59e6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:52.873694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b19d752d465420288dae25dff8bd2fa', 'request_id': 'req-02bbba61-2f02-4126-a023-9187f4ef59e6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:52.880705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b19d752d465420288dae25dff8bd2fa', 'request_id': 'req-02bbba61-2f02-4126-a023-9187f4ef59e6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:53.021848 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] The token cache is being invalidate because project 4b19d752d465420288dae25dff8bd2fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:31:53.022136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:53.022777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02bbba61-2f02-4126-a023-9187f4ef59e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:31:53.024293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1187/2354] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:31:52 2022] DELETE /identity/v3/projects/4b19d752d465420288dae25dff8bd2fa => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:53.029905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336095-4d82-4267-9618-090e97935f34 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.030396 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336095-4d82-4267-9618-090e97935f34 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.030568 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336095-4d82-4267-9618-090e97935f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.051049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5f336095-4d82-4267-9618-090e97935f34 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:53.052277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5f336095-4d82-4267-9618-090e97935f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:53.111980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1168/2355] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:53.121825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.122107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.122379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.144629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:53.145878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:53.154146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1188/2356] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:53.159032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336095-4d82-4267-9618-090e97935f34 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:53.169690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336095-4d82-4267-9618-090e97935f34 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:53.174176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:53.174795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.175075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.175344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.175985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:31:53.176308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:53.177879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:53.190269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1169/2357] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:31:53.199184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:53.208398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f7ab8-0451-4221-bc82-0176bbb1c220 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:53.213294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:53.213925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.214293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.214566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.215217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:53.217160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:53.384427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1189/2358] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:31:53 2022] POST /identity/v3/projects => generated 357 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:53.388438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:53.406116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80262e63-ebe6-4288-b8a4-963b316fecfe admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:53.424643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:53.455081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:53.455697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.456015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.456299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.456965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:53.459172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:53.553673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1170/2359] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:31:53 2022] POST /identity/v3/users => generated 279 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:53.557916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:53.567955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc98d4c-6902-49ff-9081-94158dd8b019 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:53.572527 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:53.573241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.573523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.573797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.574434 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:53.576035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:53.586132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1190/2360] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:31:53.590946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:53.600886 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ad804-2619-4124-86cf-7120be51bcdd admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:53.606611 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701a3b2-dc04-4230-9a90-d753a6d373a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:53.612373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a3b2-dc04-4230-9a90-d753a6d373a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.612373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a3b2-dc04-4230-9a90-d753a6d373a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.612373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701a3b2-dc04-4230-9a90-d753a6d373a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.612373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a3b2-dc04-4230-9a90-d753a6d373a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:53.612373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701a3b2-dc04-4230-9a90-d753a6d373a8 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:53.619531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1171/2361] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:31:53.624126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:53.634316 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9ec9f-2d82-473e-839f-ac1025f4679a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:53.638904 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:53.639545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.639824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.640122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] PATH_INFO: `/v3/projects/b3112e8e73d74fa2bb7d9055b1f8f58f/users/8e768fc6514045fca2d1d82e46111927/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.654735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3112e8e73d74fa2bb7d9055b1f8f58f, user_id=8e768fc6514045fca2d1d82e46111927, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:53.657408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:53.873708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1191/2362] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:31:53 2022] PUT /identity/v3/projects/b3112e8e73d74fa2bb7d9055b1f8f58f/users/8e768fc6514045fca2d1d82e46111927/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:53.888165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:53.888165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:53.888165 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:53.912659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] MFA Rules not processed for user `8e768fc6514045fca2d1d82e46111927`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:53.914835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:53.980402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1172/2363] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:53.984692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:54.046611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a7895-11cd-436f-bd1c-88ab01751c23 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:54.051044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:54.051632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.051907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.052196 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.052807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:54.054664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:54.335927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1192/2364] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:31:53 2022] POST /identity/v3/projects => generated 357 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:31:54.340070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:54.350829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90752a1a-576c-4111-b5cd-204516ca82b3 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:54.407227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:54.407833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.408140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.408412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.409042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:54.410994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:54.563980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1173/2365] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:31:54 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:31:54.568036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:54.579002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d961ab-febc-4ec9-ad1b-f40c806ef77b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:54.583452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:54.584332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.584608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.584875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.585503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:54.587156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:54.597631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1193/2366] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:31:54.601742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:54.612031 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61b9e1-a278-4ad5-b4bf-b6a1d261e53e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:54.616474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:54.617107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.617378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.617642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] PATH_INFO: `/v3/projects/70febeb6777c40879fc4555ef1d4927d/users/71f10021cc41471f83f3b66655e8530d/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.632434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70febeb6777c40879fc4555ef1d4927d, user_id=71f10021cc41471f83f3b66655e8530d, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:54.635046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:54.753952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1174/2367] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:31:54 2022] PUT /identity/v3/projects/70febeb6777c40879fc4555ef1d4927d/users/71f10021cc41471f83f3b66655e8530d/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:31:54.758089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:54.768057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00679312-eacf-4bf0-8ebe-ba966e8b66d3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:54.787482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:54.823273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d82d2-5b9c-4716-93d4-20af1610535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:54.823892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d82d2-5b9c-4716-93d4-20af1610535a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.824199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d82d2-5b9c-4716-93d4-20af1610535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.824501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d82d2-5b9c-4716-93d4-20af1610535a admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.825125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d82d2-5b9c-4716-93d4-20af1610535a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:54.826700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d82d2-5b9c-4716-93d4-20af1610535a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:54.836648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1194/2368] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:31:54.840900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:54.846214 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:31:54.851669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b714b28-2885-4d34-94e1-2e46df065594 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:54.857204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:54.857837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.858142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.858413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] PATH_INFO: `/v3/projects/70febeb6777c40879fc4555ef1d4927d/users/71f10021cc41471f83f3b66655e8530d/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.860428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70febeb6777c40879fc4555ef1d4927d, user_id=71f10021cc41471f83f3b66655e8530d, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:54.863022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:54.873564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1175/2369] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:31:54 2022] PUT /identity/v3/projects/70febeb6777c40879fc4555ef1d4927d/users/71f10021cc41471f83f3b66655e8530d/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:31:54.890933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:54.891217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:54.891486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:54.921296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] MFA Rules not processed for user `71f10021cc41471f83f3b66655e8530d`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:31:54.922541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:54.989017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1195/2370] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:31:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:31:55.001556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:55.003269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:31:55.060904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80944814-4646-4b13-847e-c1e8c6a47f16 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:55.065068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:55.065715 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:55.065993 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:55.066264 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:55.075184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:55.076637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:55.086783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1176/2371] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:31:55.477447 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:31:55.486847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429ecf1-e4f8-4da9-bd0e-628b1dde2458 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:31:55.490353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:31:55.490932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:31:55.491202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:31:55.491465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:31:55.545821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:31:55.547337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:31:55.555912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1196/2372] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:31:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:05.473592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:05.611094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=80 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:19.815024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:19.816158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:19.877199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cba87e-3e4c-4c46-b1cb-f9590b479d89 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:19.890575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:19.890575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:19.890575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:19.890575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:19.891038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:19.948379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:19.950659 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68db40a-d6d4-418e-a4b2-92413bf6f487 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:19.960974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1177/2373] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:21.256931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:21.267923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd8e42-26a2-4a36-8720-dcd52b237d9b service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:21.272557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:21.273252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:21.273593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:21.273936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] PATH_INFO: `/v3/users/8e768fc6514045fca2d1d82e46111927` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:21.275597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e768fc6514045fca2d1d82e46111927)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:21.277611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:21.459917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e768fc6514045fca2d1d82e46111927'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:21.460675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e768fc6514045fca2d1d82e46111927'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:21.461301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e768fc6514045fca2d1d82e46111927'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:21.469885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e768fc6514045fca2d1d82e46111927'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:21.541746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e768fc6514045fca2d1d82e46111927'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:21.560990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1197/2374] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:21 2022] DELETE /identity/v3/users/8e768fc6514045fca2d1d82e46111927 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:21.565687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:21.566158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:21.566286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:21.586591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:21.587855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:21.654327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1178/2375] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:21.663771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:21.665359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:21.725839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29c6eaf-18d5-450a-b53d-13e6d4d741aa admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:21.729475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:21.730079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:21.730355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:21.730624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:21.739936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:21.741264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:21.750767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1198/2376] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:22.326650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:22.328984 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:22.346676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d752d65-635e-4c5e-bce5-9356b3194dca None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:22.351955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:22.352626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:22.353133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:22.353346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] PATH_INFO: `/v3/projects/b3112e8e73d74fa2bb7d9055b1f8f58f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:22.354527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3112e8e73d74fa2bb7d9055b1f8f58f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:22.357727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:22.539814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:22.542938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3112e8e73d74fa2bb7d9055b1f8f58f', 'request_id': 'req-b399b8d1-6907-40fc-abed-ed998729827d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:22.549913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3112e8e73d74fa2bb7d9055b1f8f58f', 'request_id': 'req-b399b8d1-6907-40fc-abed-ed998729827d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:22.556237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3112e8e73d74fa2bb7d9055b1f8f58f', 'request_id': 'req-b399b8d1-6907-40fc-abed-ed998729827d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:22.667693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] The token cache is being invalidate because project b3112e8e73d74fa2bb7d9055b1f8f58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:22.668813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:22.669008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:22.670564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1179/2377] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:22 2022] DELETE /identity/v3/projects/b3112e8e73d74fa2bb7d9055b1f8f58f => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:22.674479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:22.686404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66a3de-d1ff-4ac2-a6e6-97f7f36b743e service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:22.745211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:22.745772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:22.746046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:22.746319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] PATH_INFO: `/v3/users/71f10021cc41471f83f3b66655e8530d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:22.748046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f10021cc41471f83f3b66655e8530d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:22.749956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:22.765949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:23.084289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f10021cc41471f83f3b66655e8530d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:23.085127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f10021cc41471f83f3b66655e8530d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:23.085914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f10021cc41471f83f3b66655e8530d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:23.096978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f10021cc41471f83f3b66655e8530d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:23.254229 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c7e69439-6f8c-40b8-b7a8-dedd62c3ee00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f10021cc41471f83f3b66655e8530d'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:23.268633 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1199/2378] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:22 2022] DELETE /identity/v3/users/71f10021cc41471f83f3b66655e8530d => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:23.897103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:23.899313 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:23.921577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b399b8d1-6907-40fc-abed-ed998729827d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:23.981496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:23.982133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:23.982408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:23.982679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] PATH_INFO: `/v3/projects/70febeb6777c40879fc4555ef1d4927d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:23.983805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70febeb6777c40879fc4555ef1d4927d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:23.985821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:24.190025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70febeb6777c40879fc4555ef1d4927d', 'request_id': 'req-8389faf3-0397-46c5-9f6e-c09f2d940598'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:24.196355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70febeb6777c40879fc4555ef1d4927d', 'request_id': 'req-8389faf3-0397-46c5-9f6e-c09f2d940598'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:24.202621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70febeb6777c40879fc4555ef1d4927d', 'request_id': 'req-8389faf3-0397-46c5-9f6e-c09f2d940598'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:24.387733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] The token cache is being invalidate because project 70febeb6777c40879fc4555ef1d4927d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:24.388175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:24.388878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8389faf3-0397-46c5-9f6e-c09f2d940598 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:24.390697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1180/2379] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:23 2022] DELETE /identity/v3/projects/70febeb6777c40879fc4555ef1d4927d => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:24.396418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:24.396877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:24.397372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:24.418694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:24.420474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:24.496182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1200/2380] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:24.498922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:24.499207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:24.500233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:24.521376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:24.522584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:24.531257 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1181/2381] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:24.535649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:24.546947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9637151-0695-4869-9d90-192fa55cecc8 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:24.552015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:24.552647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:24.552922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:24.553246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:24.553893 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:32:24.554191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:24.555786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:24.569600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1201/2382] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:32:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:32:24.574390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:24.586812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86242a3b-1069-453b-b2b7-fd2304bb49ae None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:24.592702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:24.593380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:24.593727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:24.594087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:24.594781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:24.596836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:24.757787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1182/2383] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:32:24 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:24.758916 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:24.769659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c919d3a2-bed9-4934-9b05-b22c6a7c0989 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:24.831109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:24.831708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:24.831980 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:24.832272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:24.832901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:24.835102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:24.974926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1202/2384] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:32:24 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:24.975752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:24.986170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb09d82-d24f-4946-a329-09d602ee9a19 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:24.991159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:24.991899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:24.992207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:24.992478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:24.993120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:24.994903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.006159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1183/2385] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:25.010223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.022385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706376cc-8a10-4cde-b63e-be16bd8c1ed3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.027226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdde3eb-5f0b-4e61-af1e-959c39a83555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.027828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdde3eb-5f0b-4e61-af1e-959c39a83555 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.028580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdde3eb-5f0b-4e61-af1e-959c39a83555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.028911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdde3eb-5f0b-4e61-af1e-959c39a83555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.029529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdde3eb-5f0b-4e61-af1e-959c39a83555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.031121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdde3eb-5f0b-4e61-af1e-959c39a83555 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.036487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:25.042313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1203/2386] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:25.046726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.058322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d5663-dcdb-4200-b3cf-011d93350c25 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.069680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.070582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.070867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.071148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] PATH_INFO: `/v3/projects/2de66d93962a40039b9a732ead2cec18/users/be3a7603bebe494d9304ee69eebe7a64/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.085583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:25.088579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de66d93962a40039b9a732ead2cec18, user_id=be3a7603bebe494d9304ee69eebe7a64, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.092035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.183124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1184/2387] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:25 2022] PUT /identity/v3/projects/2de66d93962a40039b9a732ead2cec18/users/be3a7603bebe494d9304ee69eebe7a64/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:25.194472 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.194754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.195024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.233303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] MFA Rules not processed for user `be3a7603bebe494d9304ee69eebe7a64`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:25.234439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:25.301500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1204/2388] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:25.306168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.375434 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f324830-8316-4b5d-96c7-81d7d0d5b67a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.380665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.608555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.608555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.608555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.608555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.608555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.645425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1185/2389] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:32:25 2022] POST /identity/v3/projects => generated 387 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:25.649433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.661175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a013141-b3ca-45a3-9757-621d5923661c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.725453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.726395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.726523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.726713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.727327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.729409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.836606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1205/2390] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:32:25 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:32:25.844404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.857052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52072918-e895-4077-a41f-9dca55911827 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.863363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.864245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.864702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.865195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.866122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.867771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.881926 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1186/2391] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:25.886148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.896653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa78f-4a2d-4e5b-8461-3c62d3cf19e2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.901388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49df86-a03f-40a2-9d6b-54695285fe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.902234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49df86-a03f-40a2-9d6b-54695285fe16 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.903301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49df86-a03f-40a2-9d6b-54695285fe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.903650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49df86-a03f-40a2-9d6b-54695285fe16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.904391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49df86-a03f-40a2-9d6b-54695285fe16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.905982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49df86-a03f-40a2-9d6b-54695285fe16 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:25.916249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1206/2392] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:25.920806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:25.930749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73b997-367c-4b69-bb69-f81ef3d0502d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:25.935973 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:25.936588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:25.936864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:25.937138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] PATH_INFO: `/v3/projects/b1a0501a5e8748778f092d2a2fb0c67a/users/bfdc2654c2774171a0cdd4648849ce40/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:25.953052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a0501a5e8748778f092d2a2fb0c67a, user_id=bfdc2654c2774171a0cdd4648849ce40, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:25.955680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:26.074169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1187/2393] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:25 2022] PUT /identity/v3/projects/b1a0501a5e8748778f092d2a2fb0c67a/users/bfdc2654c2774171a0cdd4648849ce40/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:26.086495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:26.086882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:26.087232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:26.122759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] MFA Rules not processed for user `bfdc2654c2774171a0cdd4648849ce40`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:26.123919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:26.195555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:26.201237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1207/2394] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:26.213371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:26.288354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a3981-9b52-4b9d-a71c-867375cd9a94 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:26.299206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:26.299949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:26.300356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:26.300719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:26.301449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:26.309003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:26.435627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1188/2395] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:32:26 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:26.435963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:26.445466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8acb8-bdba-444c-8f8e-e5661144b02d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:26.514598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:26.515212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:26.515486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:26.515750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:26.516405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:26.518531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:26.642895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1208/2396] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:32:26 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:26.644186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:26.656176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366cd71e-8fcf-46a4-b0ba-ab01c63a5a18 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:26.661874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:26.662485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:26.662768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:26.663151 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:26.663800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:26.665561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:26.682735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1189/2397] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:26.687957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:26.698387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d0fde-903f-4926-b177-1e85520d9695 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:26.703357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:26.705056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:26.705340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:26.705617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] PATH_INFO: `/v3/projects/418a307cac154a3fa39374e398541ae7/users/a7a8a4eca101487893e266bc6887f7a8/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:26.724018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] RBAC: Authorizing `identity:create_grant(project_id=418a307cac154a3fa39374e398541ae7, user_id=a7a8a4eca101487893e266bc6887f7a8, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:26.726641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:26.891684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1209/2398] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:26 2022] PUT /identity/v3/projects/418a307cac154a3fa39374e398541ae7/users/a7a8a4eca101487893e266bc6887f7a8/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:26.895935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:26.908787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e90a827-9a8c-41ed-9ec8-92848c3e776c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:26.931490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:26.980796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843eb980-f5a2-451e-89ee-cca3989bc40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:26.982136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb980-f5a2-451e-89ee-cca3989bc40b admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:26.982412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb980-f5a2-451e-89ee-cca3989bc40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:26.982680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843eb980-f5a2-451e-89ee-cca3989bc40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:26.983328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb980-f5a2-451e-89ee-cca3989bc40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:26.984977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843eb980-f5a2-451e-89ee-cca3989bc40b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:27.002584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1190/2399] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:27.004824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:27.015545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e2fe9a-dcce-4958-bd28-a51eca7ca65f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:27.020746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:27.021415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:27.021692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:27.021956 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] PATH_INFO: `/v3/projects/418a307cac154a3fa39374e398541ae7/users/a7a8a4eca101487893e266bc6887f7a8/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:27.024034 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=418a307cac154a3fa39374e398541ae7, user_id=a7a8a4eca101487893e266bc6887f7a8, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:27.027048 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:27.054508 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1210/2400] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:27 2022] PUT /identity/v3/projects/418a307cac154a3fa39374e398541ae7/users/a7a8a4eca101487893e266bc6887f7a8/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:27.056379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:27.056847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:27.057700 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:27.101057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] MFA Rules not processed for user `a7a8a4eca101487893e266bc6887f7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:27.103036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:27.190601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1191/2401] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:27 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:32:27.201648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:27.209741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:27.272698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd2202-17bf-435f-8ea2-f08259901648 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:27.276700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:27.277499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:27.277727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:27.278053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:27.288140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:27.289670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:27.299917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1211/2402] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:27.673005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:27.674562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:27.730910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3444a-0c31-4830-ad5d-8695fe282b95 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:27.734831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:27.735461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:27.735806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:27.736114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:27.739520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:27.793739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:27.817730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:27.819302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:27.832269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1192/2403] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:29.050212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:29.062102 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e24c60-b3fd-46a0-aac6-d1db082f8a1e service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:29.066855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:29.066855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:29.067151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:29.067486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:29.069318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:29.084644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:29.086148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:29.098231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1212/2404] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:32:29.131864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:29.141294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da37f-f9d2-44b0-a205-3bb382e7f9a4 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:29.144966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:29.145594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:29.145870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:29.146144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:29.206620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:29.208079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:29.217798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1193/2405] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:29.274402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:29.276652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:29.326246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:29.348017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345eef7-b622-47b4-ad93-b766b1498991 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:29.352584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:29.356877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:29.356877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:29.356877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] PATH_INFO: `/v3/users/46dffe5f71cf4d2ca7bf42f86d2bb07a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:29.356877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46dffe5f71cf4d2ca7bf42f86d2bb07a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:29.357437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:29.732756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46dffe5f71cf4d2ca7bf42f86d2bb07a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:29.733663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46dffe5f71cf4d2ca7bf42f86d2bb07a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:29.736110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46dffe5f71cf4d2ca7bf42f86d2bb07a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:29.741552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46dffe5f71cf4d2ca7bf42f86d2bb07a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:29.890827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46dffe5f71cf4d2ca7bf42f86d2bb07a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:29.909106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1213/2406] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:29 2022] DELETE /identity/v3/users/46dffe5f71cf4d2ca7bf42f86d2bb07a => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:30.189948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:30.205557 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea634a5-40d7-4bb0-b074-e19849decdeb service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:30.284714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:30.285760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:30.286187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:30.286903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] PATH_INFO: `/v3/projects/e05759c8ab0248e58d35630a611912f0` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:30.288387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e05759c8ab0248e58d35630a611912f0)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:30.290623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:30.474392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e05759c8ab0248e58d35630a611912f0', 'request_id': 'req-645b9d83-02a7-4997-ab61-9da0091b1010'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.480620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e05759c8ab0248e58d35630a611912f0', 'request_id': 'req-645b9d83-02a7-4997-ab61-9da0091b1010'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.487575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e05759c8ab0248e58d35630a611912f0', 'request_id': 'req-645b9d83-02a7-4997-ab61-9da0091b1010'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.577036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] The token cache is being invalidate because project e05759c8ab0248e58d35630a611912f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:30.581695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.581695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.581695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1194/2407] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:30 2022] DELETE /identity/v3/projects/e05759c8ab0248e58d35630a611912f0 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:30.583885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:30.585579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:30.600151 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ed1aa0-716b-48e0-a290-9d748406881e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:30.658121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:30.658746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:30.659021 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:30.659289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] PATH_INFO: `/v3/users/8ab73f35559a4e908e15e7ae8aa5dd51` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:30.660509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab73f35559a4e908e15e7ae8aa5dd51)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:30.662480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:30.993670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab73f35559a4e908e15e7ae8aa5dd51'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.994232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab73f35559a4e908e15e7ae8aa5dd51'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:30.994852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab73f35559a4e908e15e7ae8aa5dd51'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.002212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab73f35559a4e908e15e7ae8aa5dd51'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.005856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:31.130230 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab73f35559a4e908e15e7ae8aa5dd51'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.149015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1214/2408] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:30 2022] DELETE /identity/v3/users/8ab73f35559a4e908e15e7ae8aa5dd51 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:31.417429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:31.418913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:31.432101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b9d83-02a7-4997-ab61-9da0091b1010 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:31.435910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:31.493296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:31.493932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:31.494215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:31.494494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] PATH_INFO: `/v3/projects/dbd2f7d25d2643b1a008cf3d07822038` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:31.495622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd2f7d25d2643b1a008cf3d07822038)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:31.497733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:31.627362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd2f7d25d2643b1a008cf3d07822038', 'request_id': 'req-a02ac01d-594a-4fce-a7cf-e013ad861dcc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.633903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd2f7d25d2643b1a008cf3d07822038', 'request_id': 'req-a02ac01d-594a-4fce-a7cf-e013ad861dcc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.640098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd2f7d25d2643b1a008cf3d07822038', 'request_id': 'req-a02ac01d-594a-4fce-a7cf-e013ad861dcc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.804783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] The token cache is being invalidate because project dbd2f7d25d2643b1a008cf3d07822038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:31.810265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.810265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a02ac01d-594a-4fce-a7cf-e013ad861dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:31.810265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1195/2409] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:31 2022] DELETE /identity/v3/projects/dbd2f7d25d2643b1a008cf3d07822038 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:36.934837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:36.936306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:37.002588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff664343-f8f1-43ce-acb2-32b8034d2af3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:37.063199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:37.063858 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:37.064167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:37.064546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:37.066384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:37.126393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:37.127859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:37.137066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1215/2410] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:37.174179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:37.174664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:37.174771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:37.194250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:37.195465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:37.253056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1196/2411] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:32:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:37.380735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:37.382204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:37.432880 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:37.461052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967f182-4d8a-4a47-b9c4-973bd4463a02 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:37.467079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1b3c9-02f0-45e2-becc-b1e417220980 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:37.467789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b3c9-02f0-45e2-becc-b1e417220980 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:37.468106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b3c9-02f0-45e2-becc-b1e417220980 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:37.468382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1b3c9-02f0-45e2-becc-b1e417220980 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:37.477538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b3c9-02f0-45e2-becc-b1e417220980 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:37.479159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1b3c9-02f0-45e2-becc-b1e417220980 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:37.487706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1216/2412] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:32:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:32:44.297391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:44.304738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:44.353059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:44.380113 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1db27-a712-44ce-88e9-e7f1165c1768 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:44.393420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:44.394074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:44.394353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:44.394681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] PATH_INFO: `/v3/users/be3a7603bebe494d9304ee69eebe7a64` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:44.395946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3a7603bebe494d9304ee69eebe7a64)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:44.397895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:44.671680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3a7603bebe494d9304ee69eebe7a64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:44.672580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3a7603bebe494d9304ee69eebe7a64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:44.673312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3a7603bebe494d9304ee69eebe7a64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:44.680802 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3a7603bebe494d9304ee69eebe7a64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:44.752921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3a7603bebe494d9304ee69eebe7a64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:44.776731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1197/2413] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:44 2022] DELETE /identity/v3/users/be3a7603bebe494d9304ee69eebe7a64 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:44.781968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:44.785910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:44.785910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:44.802351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:44.803694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:44.822622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:44.825349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:44.870260 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1217/2414] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:44.894475 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:44.907148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:44.920620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede34a84-3650-4b20-9f5a-c34c912b380b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:44.928033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:44.929511 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:44.929761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:44.930160 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:44.932736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:45.027103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399dd3b4-d8b4-482b-8e48-2ab830613eea None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:45.033390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:45.034138 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:45.034528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:45.034892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:45.058594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:45.061562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:45.069998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:45.077038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1218/2415] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:45.107103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:45.109253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:45.136047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1198/2416] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:44 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:32:45.936762 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:45.938241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:45.951041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8f0d7-4cdf-453f-95d2-789e8e74793c service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:45.955778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:45.956426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:45.956699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:45.956964 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] PATH_INFO: `/v3/projects/2de66d93962a40039b9a732ead2cec18` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:45.958109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de66d93962a40039b9a732ead2cec18)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:45.960410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:46.144862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de66d93962a40039b9a732ead2cec18', 'request_id': 'req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.151545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de66d93962a40039b9a732ead2cec18', 'request_id': 'req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.159199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de66d93962a40039b9a732ead2cec18', 'request_id': 'req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.264043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] The token cache is being invalidate because project 2de66d93962a40039b9a732ead2cec18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:46.264525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.265413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.267300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1219/2417] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:45 2022] DELETE /identity/v3/projects/2de66d93962a40039b9a732ead2cec18 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:46.271280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:46.285831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9023a3c-994e-4be6-baf6-72d84863fd52 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:46.521670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:46.522388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:46.522727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:46.523071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] PATH_INFO: `/v3/users/bfdc2654c2774171a0cdd4648849ce40` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:46.524314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfdc2654c2774171a0cdd4648849ce40)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:46.526726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:46.831438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfdc2654c2774171a0cdd4648849ce40'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.832304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfdc2654c2774171a0cdd4648849ce40'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.833028 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfdc2654c2774171a0cdd4648849ce40'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:46.841650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfdc2654c2774171a0cdd4648849ce40'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:47.032962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfdc2654c2774171a0cdd4648849ce40'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:47.059373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1199/2418] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:46 2022] DELETE /identity/v3/users/bfdc2654c2774171a0cdd4648849ce40 => generated 0 bytes in 788 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:47.689004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:47.691189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:47.710966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:47.726523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdc6e5-2f37-4e15-8a27-e7b96f398f88 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:47.827510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:47.828328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:47.828611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:47.828877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] PATH_INFO: `/v3/projects/b1a0501a5e8748778f092d2a2fb0c67a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:47.830015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a0501a5e8748778f092d2a2fb0c67a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:47.831958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:48.003566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a0501a5e8748778f092d2a2fb0c67a', 'request_id': 'req-56fa6fc7-cdea-4e08-8e85-b9f730482b89'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.014807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a0501a5e8748778f092d2a2fb0c67a', 'request_id': 'req-56fa6fc7-cdea-4e08-8e85-b9f730482b89'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.024896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a0501a5e8748778f092d2a2fb0c67a', 'request_id': 'req-56fa6fc7-cdea-4e08-8e85-b9f730482b89'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.163201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] The token cache is being invalidate because project b1a0501a5e8748778f092d2a2fb0c67a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:48.163734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.165867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.167582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1220/2419] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:47 2022] DELETE /identity/v3/projects/b1a0501a5e8748778f092d2a2fb0c67a => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:48.174290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:48.175720 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:48.191582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8995d3-0c21-46c0-9100-752aaece0a0f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:48.257898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:48.272726 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:48.273515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:48.273797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:48.274069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] PATH_INFO: `/v3/users/a7a8a4eca101487893e266bc6887f7a8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:48.275211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a8a4eca101487893e266bc6887f7a8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:48.277382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:48.623369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a8a4eca101487893e266bc6887f7a8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.625302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a8a4eca101487893e266bc6887f7a8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.626311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a8a4eca101487893e266bc6887f7a8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.638651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a8a4eca101487893e266bc6887f7a8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.712630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-edf411b1-8eac-4817-90d8-7a940c044eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a8a4eca101487893e266bc6887f7a8'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:48.735494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1200/2420] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:32:48 2022] DELETE /identity/v3/users/a7a8a4eca101487893e266bc6887f7a8 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:49.457015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:49.458551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:49.471855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa6fc7-cdea-4e08-8e85-b9f730482b89 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:49.535621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:49.536294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:49.536585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:49.536835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] PATH_INFO: `/v3/projects/418a307cac154a3fa39374e398541ae7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:49.537979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=418a307cac154a3fa39374e398541ae7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:49.539901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:49.669961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52dae9-208c-4609-a097-e51c1514c471 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:49.670490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52dae9-208c-4609-a097-e51c1514c471 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:49.670490 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52dae9-208c-4609-a097-e51c1514c471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:49.729153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:49.730156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0d52dae9-208c-4609-a097-e51c1514c471 None None] MFA Rules not processed for user `0782e27e2318438fbf155dfa9cb834b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:49.731588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0d52dae9-208c-4609-a097-e51c1514c471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:49.747984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '418a307cac154a3fa39374e398541ae7', 'request_id': 'req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:49.757931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '418a307cac154a3fa39374e398541ae7', 'request_id': 'req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:49.768640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '418a307cac154a3fa39374e398541ae7', 'request_id': 'req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:49.811400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1201/2421] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:32:49.972614 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] The token cache is being invalidate because project 418a307cac154a3fa39374e398541ae7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:32:49.973058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:49.973810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-06bf7ea9-5637-4ccd-8ecc-bd0fe308e861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:32:49.975372 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1221/2422] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:32:49 2022] DELETE /identity/v3/projects/418a307cac154a3fa39374e398541ae7 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:49.982347 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:49.982855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:49.983880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.009847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:50.011240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:50.072947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:50.135480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1202/2423] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:50.144836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.145197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.145792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.174578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:50.175947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:50.186150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1222/2424] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:50.190541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:50.205812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ebd58-cfb0-4c12-9d62-8ce7ba0893d1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:50.211604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:50.212398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.212815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.213186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.213946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:32:50.214365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:50.217508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:50.236051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1203/2425] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:32:50.239416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:50.256866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145ab62-cc19-4483-8d03-6121dfb7532f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:50.268236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:50.272311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.272311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.272311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.272311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:50.272311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:50.529153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1223/2426] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:32:50 2022] POST /identity/v3/projects => generated 407 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:50.530364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:50.539927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a5ed3-d776-457d-b6e8-f2dd10dca89d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:50.606278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:50.607007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.607287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.607558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.608219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:50.610112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:50.748797 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1204/2427] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:32:50 2022] POST /identity/v3/users => generated 304 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:32:50.752911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:50.764507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32284f2a-e42f-46b3-8ef6-f0b535880369 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:50.771231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:50.771912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.772300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.772630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.773333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:50.775116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:50.788090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1224/2428] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:32:50.792034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:50.810464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b42e7-9252-462f-b8d1-a130c521807b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:50.816415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49659fe-c400-4b86-a54e-3f9b94be254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:50.817095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49659fe-c400-4b86-a54e-3f9b94be254a admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.817451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49659fe-c400-4b86-a54e-3f9b94be254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.817718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49659fe-c400-4b86-a54e-3f9b94be254a admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.818348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49659fe-c400-4b86-a54e-3f9b94be254a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:50.819979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49659fe-c400-4b86-a54e-3f9b94be254a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:50.831121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1205/2429] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:50.835473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:50.849636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a85eb-0624-4f29-af78-23d6f2802a80 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:50.855930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:50.855930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.855930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.855930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] PATH_INFO: `/v3/projects/e9bb563c110b47b2b7a9adaf3962a2b5/users/470da8cfd22d4b3bbfb0a4aad0fc5938/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:50.871267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bb563c110b47b2b7a9adaf3962a2b5, user_id=470da8cfd22d4b3bbfb0a4aad0fc5938, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:50.874326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:50.987357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1225/2430] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:50 2022] PUT /identity/v3/projects/e9bb563c110b47b2b7a9adaf3962a2b5/users/470da8cfd22d4b3bbfb0a4aad0fc5938/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:50.998707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:50.999033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:50.999311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.029664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] MFA Rules not processed for user `470da8cfd22d4b3bbfb0a4aad0fc5938`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:51.032114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:51.098778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1206/2431] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:50 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:51.103243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:51.166419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ecdad-1d90-4768-ba61-7a9a753cf0a6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:51.171408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:51.172117 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.172309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.172579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.173205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:51.175167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:51.316430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1226/2432] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:32:51 2022] POST /identity/v3/projects => generated 407 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:32:51.320176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:51.334907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be90b7-1646-49bd-8a09-a269653bad71 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:51.377667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:51.417998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:51.419820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.420198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.420596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.421682 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:51.424650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:51.532902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1207/2433] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:32:51 2022] POST /identity/v3/users => generated 310 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:32:51.534769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:51.545826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3e11-e46a-4b0b-a5dc-291351244bcd admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:51.550818 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:51.551782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.552338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.552821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.553680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:51.555433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:51.569013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1227/2434] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:51.576209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:51.586612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd644a86-6933-4b61-941e-ae0671ea43f0 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:51.591610 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:51.592842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.593124 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.593398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] PATH_INFO: `/v3/projects/dc447c98152c4eea979bdc53d17bb676/users/07f72a79c980404aaeab2328377f2528/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.615566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc447c98152c4eea979bdc53d17bb676, user_id=07f72a79c980404aaeab2328377f2528, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:51.618399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:51.753848 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1208/2435] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:51 2022] PUT /identity/v3/projects/dc447c98152c4eea979bdc53d17bb676/users/07f72a79c980404aaeab2328377f2528/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:51.758104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:51.768421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052064-2351-4a68-8145-f58f41b30091 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:51.801790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:32:51.826280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e7c4a-bf4f-46c5-a9a3-1392c4d07b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:51.826889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7c4a-bf4f-46c5-a9a3-1392c4d07b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.827167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7c4a-bf4f-46c5-a9a3-1392c4d07b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.827437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e7c4a-bf4f-46c5-a9a3-1392c4d07b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.828103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7c4a-bf4f-46c5-a9a3-1392c4d07b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:51.829705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e7c4a-bf4f-46c5-a9a3-1392c4d07b5d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:51.842533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1228/2436] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:32:51.845150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:51.856207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e357f42-ebab-493b-8eda-89a9a7758898 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:51.860944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:51.861683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.862025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.862365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] PATH_INFO: `/v3/projects/dc447c98152c4eea979bdc53d17bb676/users/07f72a79c980404aaeab2328377f2528/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.864326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc447c98152c4eea979bdc53d17bb676, user_id=07f72a79c980404aaeab2328377f2528, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:51.867054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:51.877970 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1209/2437] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:32:51 2022] PUT /identity/v3/projects/dc447c98152c4eea979bdc53d17bb676/users/07f72a79c980404aaeab2328377f2528/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:32:51.892423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:51.892763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:51.893070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:51.924750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] MFA Rules not processed for user `07f72a79c980404aaeab2328377f2528`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:32:51.925963 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:51.994505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1229/2438] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:32:51 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:32:52.005761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:52.008824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:52.065920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14c41e4-097a-4b94-b326-2037dc9d7fb7 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:52.072205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49093c03-ba94-4322-9dc0-903e83caeeea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:52.072205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093c03-ba94-4322-9dc0-903e83caeeea service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:52.072205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093c03-ba94-4322-9dc0-903e83caeeea service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:52.072205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49093c03-ba94-4322-9dc0-903e83caeeea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:52.080908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49093c03-ba94-4322-9dc0-903e83caeeea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:52.082367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49093c03-ba94-4322-9dc0-903e83caeeea service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:52.092268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1210/2439] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:52 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:32:52.510401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:32:52.513373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:32:52.525692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388eeb2-ddd1-4105-81be-e8b9c729b334 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:32:52.530331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e2267-7e9f-4eba-b6b1-f2a59b0495d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:32:52.530999 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2267-7e9f-4eba-b6b1-f2a59b0495d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:32:52.531252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2267-7e9f-4eba-b6b1-f2a59b0495d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:32:52.531526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e2267-7e9f-4eba-b6b1-f2a59b0495d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:32:52.594300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e2267-7e9f-4eba-b6b1-f2a59b0495d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:32:52.595794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e2267-7e9f-4eba-b6b1-f2a59b0495d6 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:32:52.606358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1230/2440] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:32:52 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:01.269146 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9164bd4-a245-47f8-ab9c-5f332435f96a None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:01.272822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9164bd4-a245-47f8-ab9c-5f332435f96a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:01.272822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9164bd4-a245-47f8-ab9c-5f332435f96a None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:01.272822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1211/2441] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:33:01.277170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a5b9e-df22-48fb-bd82-f376aca5fe48 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:01.278852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a5b9e-df22-48fb-bd82-f376aca5fe48 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:01.278852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6a5b9e-df22-48fb-bd82-f376aca5fe48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:01.335326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5d6a5b9e-df22-48fb-bd82-f376aca5fe48 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:01.335326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5d6a5b9e-df22-48fb-bd82-f376aca5fe48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:01.397859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=100 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:01.421637 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1231/2442] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:33:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:01.450426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9164bd4-a245-47f8-ab9c-5f332435f96a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:01.450426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b9164bd4-a245-47f8-ab9c-5f332435f96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:01.496774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9164bd4-a245-47f8-ab9c-5f332435f96a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:01.500322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38e594-a567-41b3-a340-e8c0d3d656d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:01.500958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38e594-a567-41b3-a340-e8c0d3d656d0 service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:01.501196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38e594-a567-41b3-a340-e8c0d3d656d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:01.501464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38e594-a567-41b3-a340-e8c0d3d656d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:01.511342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38e594-a567-41b3-a340-e8c0d3d656d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:01.513514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38e594-a567-41b3-a340-e8c0d3d656d0 service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:01.524438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1212/2443] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:01.840476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3394a17-249b-4855-9382-c62e96a3ea1b None None] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:01.840938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3394a17-249b-4855-9382-c62e96a3ea1b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:01.841218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3394a17-249b-4855-9382-c62e96a3ea1b None None] PATH_INFO: `/` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:01.842585 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1232/2444] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:33:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 10 11:33:01.847378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8ee63-e1b4-4e30-bafe-5a67e61ef937 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:01.847660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8ee63-e1b4-4e30-bafe-5a67e61ef937 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:01.847927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8ee63-e1b4-4e30-bafe-5a67e61ef937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:01.885083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f0e8ee63-e1b4-4e30-bafe-5a67e61ef937 None None] MFA Rules not processed for user `52773c21d6e9406086113a9fd74d5f62`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:01.886775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f0e8ee63-e1b4-4e30-bafe-5a67e61ef937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:01.898292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:01.944097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1213/2445] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:33:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:01.949408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3394a17-249b-4855-9382-c62e96a3ea1b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:01.951309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b3394a17-249b-4855-9382-c62e96a3ea1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:01.969515 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3394a17-249b-4855-9382-c62e96a3ea1b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:01.973065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae0d30-9503-46d5-b01b-e58aad525681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:01.973711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae0d30-9503-46d5-b01b-e58aad525681 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:01.973992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae0d30-9503-46d5-b01b-e58aad525681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:01.974258 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae0d30-9503-46d5-b01b-e58aad525681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:01.982154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae0d30-9503-46d5-b01b-e58aad525681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:01.983556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ae0d30-9503-46d5-b01b-e58aad525681 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:01.991450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1233/2446] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:02.077055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4081e8-5e3c-4a82-8672-40218ee277cf None None] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:02.077336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4081e8-5e3c-4a82-8672-40218ee277cf None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:02.077605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4081e8-5e3c-4a82-8672-40218ee277cf None None] PATH_INFO: `/` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:02.079082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1214/2447] 10.222.0.13 () {54 vars in 921 bytes} [Wed Aug 10 11:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 10 11:33:02.084333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239a5e-45e5-42cb-9c4e-f31e5c077119 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:02.084621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239a5e-45e5-42cb-9c4e-f31e5c077119 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:02.085160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77239a5e-45e5-42cb-9c4e-f31e5c077119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:02.104436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-77239a5e-45e5-42cb-9c4e-f31e5c077119 None None] MFA Rules not processed for user `462638e8386c49668290954c0211728c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:02.106526 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-77239a5e-45e5-42cb-9c4e-f31e5c077119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:02.115706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1234/2448] 10.222.0.13 () {58 vars in 1018 bytes} [Wed Aug 10 11:33:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:02.127132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4081e8-5e3c-4a82-8672-40218ee277cf None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:02.128841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7d4081e8-5e3c-4a82-8672-40218ee277cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:02.142607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4081e8-5e3c-4a82-8672-40218ee277cf None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:02.147641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:02.148269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:02.148543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:02.148840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:02.159931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:02.161422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:02.172820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1215/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:05.296096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:05.299482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:05.299482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:05.345445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:05.346639 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:05.411847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1235/2450] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:33:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:05.429586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:05.430994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:05.498978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:05.523418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03df0dc-7aa3-4e19-9dc9-b8d1156fe395 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:05.528932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb03fe-d8a1-4d2a-922b-1ddb27bbb3a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:05.529601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb03fe-d8a1-4d2a-922b-1ddb27bbb3a7 service swift] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:05.529871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb03fe-d8a1-4d2a-922b-1ddb27bbb3a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:05.530180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb03fe-d8a1-4d2a-922b-1ddb27bbb3a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:05.538458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb03fe-d8a1-4d2a-922b-1ddb27bbb3a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:05.540122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb03fe-d8a1-4d2a-922b-1ddb27bbb3a7 service swift] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:05.548866 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1216/2451] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:33:05.564652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008144f-badf-4fed-8f06-77ebcf5c6dee None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:05.569525 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:05.570166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:05.570442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:05.570708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] PATH_INFO: `/v3/users/470da8cfd22d4b3bbfb0a4aad0fc5938` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:05.571825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=470da8cfd22d4b3bbfb0a4aad0fc5938)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:05.573896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:05.737214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:05.876573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '470da8cfd22d4b3bbfb0a4aad0fc5938'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:05.877214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '470da8cfd22d4b3bbfb0a4aad0fc5938'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:05.877809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '470da8cfd22d4b3bbfb0a4aad0fc5938'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:05.884757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '470da8cfd22d4b3bbfb0a4aad0fc5938'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:05.991407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '470da8cfd22d4b3bbfb0a4aad0fc5938'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:06.015740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1236/2452] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:05 2022] DELETE /identity/v3/users/470da8cfd22d4b3bbfb0a4aad0fc5938 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:33:06.020632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:06.021320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:06.021835 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:06.056692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:06.058003 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:06.157014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1217/2453] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:06.174065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:06.174065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:06.260058 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bdbe06-07e3-459f-b958-d90f1cc494d5 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:06.264672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:06.265333 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:06.265827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:06.266112 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:06.283524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:06.285020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:06.296253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1237/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:06.829111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:06.829111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:06.847446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba000c68-c98a-46ad-a3c5-a02de4fda36e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:06.853405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:06.854159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:06.854446 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:06.854724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] PATH_INFO: `/v3/projects/e9bb563c110b47b2b7a9adaf3962a2b5` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:06.855937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bb563c110b47b2b7a9adaf3962a2b5)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:06.857898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:06.881840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:07.063104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bb563c110b47b2b7a9adaf3962a2b5', 'request_id': 'req-0b82b969-5edb-4e14-9b29-15540dfbea05'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.069495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bb563c110b47b2b7a9adaf3962a2b5', 'request_id': 'req-0b82b969-5edb-4e14-9b29-15540dfbea05'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.075982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bb563c110b47b2b7a9adaf3962a2b5', 'request_id': 'req-0b82b969-5edb-4e14-9b29-15540dfbea05'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.192029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] The token cache is being invalidate because project e9bb563c110b47b2b7a9adaf3962a2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:07.192414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.193121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.194593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1218/2455] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:06 2022] DELETE /identity/v3/projects/e9bb563c110b47b2b7a9adaf3962a2b5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:07.198584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:07.209158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32eb16-28c3-4848-981b-cf248887038c service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:07.306340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:07.307277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:07.307401 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:07.307671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] PATH_INFO: `/v3/users/07f72a79c980404aaeab2328377f2528` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:07.309379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f72a79c980404aaeab2328377f2528)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:07.311510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:07.555451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f72a79c980404aaeab2328377f2528'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.556128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f72a79c980404aaeab2328377f2528'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.556845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f72a79c980404aaeab2328377f2528'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.559601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:07.565365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f72a79c980404aaeab2328377f2528'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.649307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-30e2513d-c457-4c13-997c-b6c9742d43ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f72a79c980404aaeab2328377f2528'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:07.668419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1238/2456] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:07 2022] DELETE /identity/v3/users/07f72a79c980404aaeab2328377f2528 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:08.214842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:08.217285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:08.242515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82b969-5edb-4e14-9b29-15540dfbea05 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:08.318156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:08.318901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:08.319404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:08.319823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] PATH_INFO: `/v3/projects/dc447c98152c4eea979bdc53d17bb676` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:08.321211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc447c98152c4eea979bdc53d17bb676)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:08.326599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:08.470300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc447c98152c4eea979bdc53d17bb676', 'request_id': 'req-fdd1eb46-f1fa-40a7-8d04-2901946a47de'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:08.478790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc447c98152c4eea979bdc53d17bb676', 'request_id': 'req-fdd1eb46-f1fa-40a7-8d04-2901946a47de'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:08.486975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc447c98152c4eea979bdc53d17bb676', 'request_id': 'req-fdd1eb46-f1fa-40a7-8d04-2901946a47de'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:08.615163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] The token cache is being invalidate because project dc447c98152c4eea979bdc53d17bb676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:08.615722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:08.616385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fdd1eb46-f1fa-40a7-8d04-2901946a47de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:08.617915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1219/2457] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:08 2022] DELETE /identity/v3/projects/dc447c98152c4eea979bdc53d17bb676 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:08.622993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6010373-a2d8-448e-8607-32100494677d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:08.623587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6010373-a2d8-448e-8607-32100494677d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:08.623587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6010373-a2d8-448e-8607-32100494677d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:08.645197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f6010373-a2d8-448e-8607-32100494677d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:08.647022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f6010373-a2d8-448e-8607-32100494677d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:08.729177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1239/2458] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:33:08.787913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:08.788232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:08.788507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:08.809385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:08.810603 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:08.818373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1220/2459] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:08.822632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6010373-a2d8-448e-8607-32100494677d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:08.833404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6010373-a2d8-448e-8607-32100494677d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:08.838364 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:08.838975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:08.839250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:08.839521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:08.840582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:33:08.841516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:08.843123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:08.858661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1240/2460] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:33:08.863163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:08.872721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b526e6a-b1e9-4d3c-a012-602dacea8880 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:08.877351 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:08.877956 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:08.878233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:08.878504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:08.879119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:08.881060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:09.053723 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1221/2461] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:33:08 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:09.057359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:09.069465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12851c29-e5a6-476f-b576-64701131e370 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:09.128492 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:09.129232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:09.129578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:09.129914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:09.130616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:09.132779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:09.270999 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1241/2462] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:33:09 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:09.272229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:09.290976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12903c4d-df0e-407f-a5f1-4912e76d7f82 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:09.296540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:09.297153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:09.297427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:09.297694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:09.298334 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:09.299915 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:09.302297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:09.311122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1222/2463] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:09.315343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:09.325999 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528aec8-f858-4b31-b31f-e74ae9fd6355 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:09.331518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58da73b-6c02-4fdf-a1e6-1b8222050b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:09.332376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da73b-6c02-4fdf-a1e6-1b8222050b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:09.332805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da73b-6c02-4fdf-a1e6-1b8222050b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:09.333236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58da73b-6c02-4fdf-a1e6-1b8222050b36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:09.334631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58da73b-6c02-4fdf-a1e6-1b8222050b36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:09.336783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58da73b-6c02-4fdf-a1e6-1b8222050b36 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:09.349793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1242/2464] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:09.354044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:09.365194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7de1d-8d8e-4caf-83a2-6679c339c348 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:09.381185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:09.384166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:09.384166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:09.384166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] PATH_INFO: `/v3/projects/1011efd26c7d43e1b29fc653c7d3a6c7/users/0ad1af9764774eb19edefdd88c24b1d3/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:09.401125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1011efd26c7d43e1b29fc653c7d3a6c7, user_id=0ad1af9764774eb19edefdd88c24b1d3, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:09.404947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:09.533481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1223/2465] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:09 2022] PUT /identity/v3/projects/1011efd26c7d43e1b29fc653c7d3a6c7/users/0ad1af9764774eb19edefdd88c24b1d3/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:09.543696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:09.543978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:09.544330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:09.573530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] MFA Rules not processed for user `0ad1af9764774eb19edefdd88c24b1d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:09.575711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:09.605171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:09.658041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1243/2466] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:09.662606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:09.783717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159d733a-a6ae-4193-a002-088565af728a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:09.788585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:09.789203 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:09.789483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:09.790231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:09.790481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:09.792508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:09.937025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1224/2467] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:33:09 2022] POST /identity/v3/projects => generated 383 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:33:09.941084 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:09.953248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0741ba68-084d-451d-9e00-e9e4907b1338 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.014094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.014867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.015216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.015480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.016441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:10.018940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:10.126246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1244/2468] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:33:09 2022] POST /identity/v3/users => generated 292 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:10.129888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:10.141531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad48351-aeeb-4a53-bd3b-7abf6bd1fabc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.146342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.147018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.147346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.147680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.148443 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:10.150091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:10.161500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1225/2469] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:33:10.166302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:10.177843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c800f72-ff09-4565-aeb3-e769d39ac01a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.185306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec815ce7-5e77-4309-b370-568a55e6c9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.186079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815ce7-5e77-4309-b370-568a55e6c9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.186450 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815ce7-5e77-4309-b370-568a55e6c9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.186812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec815ce7-5e77-4309-b370-568a55e6c9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.187714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec815ce7-5e77-4309-b370-568a55e6c9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:10.189437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec815ce7-5e77-4309-b370-568a55e6c9e0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:10.204914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1245/2470] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:10.206049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:10.219216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42df993-395b-4ef4-93e5-9c104eaf4e11 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.223903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.231869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.231869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.231869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] PATH_INFO: `/v3/projects/f291fa1c288b4ed797997f5a2b2308a8/users/12acfa06095e43419eb8ba09ce6ee3ec/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.242776 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f291fa1c288b4ed797997f5a2b2308a8, user_id=12acfa06095e43419eb8ba09ce6ee3ec, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:10.245595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:10.361235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1226/2471] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:10 2022] PUT /identity/v3/projects/f291fa1c288b4ed797997f5a2b2308a8/users/12acfa06095e43419eb8ba09ce6ee3ec/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:33:10.374478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.374830 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.375174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.408086 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] MFA Rules not processed for user `12acfa06095e43419eb8ba09ce6ee3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:10.409824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:10.485625 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1246/2472] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:10.489887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:10.565460 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827acf2a-3a48-4a0d-b479-92b9e23cd851 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.570373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.571050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.571328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.571607 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.572444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:10.575215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:10.737792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1227/2473] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:33:10 2022] POST /identity/v3/projects => generated 383 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:10.741706 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:10.756017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775495a3-d2be-47d0-b8ff-d658b79a08b3 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.788537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:10.821306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.822008 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:10.822388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:10.822721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:10.823403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:10.826656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:10.968047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1247/2474] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:33:10 2022] POST /identity/v3/users => generated 298 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:10.971471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:10.987604 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fbbdc6-fe2c-4e3e-a0e1-c18e5dbc807c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:10.998538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:10.999093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:11.000025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:11.000305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:11.000960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:11.002562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:11.005131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:11.016576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1228/2475] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:33:11.020874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:11.033990 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b499e6-83e1-4b83-9ba7-134b537b2ab4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:11.042232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:11.042232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:11.042232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:11.042232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] PATH_INFO: `/v3/projects/ac2577745b5f4109abc8c715e7f3ae95/users/b128fe9da597453e8921507dab605f56/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:11.065844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2577745b5f4109abc8c715e7f3ae95, user_id=b128fe9da597453e8921507dab605f56, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:11.068667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:11.187178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1248/2476] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:11 2022] PUT /identity/v3/projects/ac2577745b5f4109abc8c715e7f3ae95/users/b128fe9da597453e8921507dab605f56/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:33:11.191173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:11.202664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67c5a0-05c5-41c1-b969-1244f561a860 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:11.263105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549de15-5b91-4c6c-bb93-50c944abdb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:11.263672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549de15-5b91-4c6c-bb93-50c944abdb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:11.263950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549de15-5b91-4c6c-bb93-50c944abdb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:11.264249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549de15-5b91-4c6c-bb93-50c944abdb6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:11.264895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549de15-5b91-4c6c-bb93-50c944abdb6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:11.266489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d549de15-5b91-4c6c-bb93-50c944abdb6d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:11.282353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1229/2477] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:11.283822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:11.295231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158ff55-c825-4d8e-90b8-68c6b6da579b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:11.300325 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:11.300993 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:11.301272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:11.301819 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] PATH_INFO: `/v3/projects/ac2577745b5f4109abc8c715e7f3ae95/users/b128fe9da597453e8921507dab605f56/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:11.303867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2577745b5f4109abc8c715e7f3ae95, user_id=b128fe9da597453e8921507dab605f56, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:11.306553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:11.317749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1249/2478] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:11 2022] PUT /identity/v3/projects/ac2577745b5f4109abc8c715e7f3ae95/users/b128fe9da597453e8921507dab605f56/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:11.329163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:11.329554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:11.329829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:11.361549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] MFA Rules not processed for user `b128fe9da597453e8921507dab605f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:11.362810 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:11.446504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1230/2479] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:33:11.459567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:11.461647 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:11.536253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a98a6-deb3-493e-b68e-85c9ee8f2ee6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:11.548379 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:11.549785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:11.550432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:11.551065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:11.648586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:11.650019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:11.660798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1250/2480] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:14.967714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:14.978758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76c86e0-ce13-4c03-b4a0-e3ded6515e22 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:14.983649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:14.991454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:14.991454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:14.991454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:15.042089 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:15.043948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:15.052670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1231/2481] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:16.433588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:16.442560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b33dd4-2009-455e-a54d-51365673a14a service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:16.446383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:16.446969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:16.447241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:16.447531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:16.457872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:16.459494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:16.469469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1251/2482] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:20.471329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:20.475440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:20.500806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:20.539505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95803c4f-9ca8-4969-b8ff-c6a6f27a63a0 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:20.543920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:20.544521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:20.544804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:20.545075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:20.546816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:20.556879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:20.558372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838e7c6-c189-4736-80bb-4ecdebcf8b32 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:20.571451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1232/2483] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:22.505812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:22.534318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:22.571439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415cad6-27cb-46f6-b878-7511edcce75c service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:22.576103 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:22.576746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:22.577024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:22.577350 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] PATH_INFO: `/v3/users/0ad1af9764774eb19edefdd88c24b1d3` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:22.578489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad1af9764774eb19edefdd88c24b1d3)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:22.580616 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:22.796698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad1af9764774eb19edefdd88c24b1d3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:22.797389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad1af9764774eb19edefdd88c24b1d3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:22.807403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad1af9764774eb19edefdd88c24b1d3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:22.815615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad1af9764774eb19edefdd88c24b1d3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:22.923615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad1af9764774eb19edefdd88c24b1d3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:22.941794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1252/2484] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:22 2022] DELETE /identity/v3/users/0ad1af9764774eb19edefdd88c24b1d3 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:22.946216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:22.946728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:22.946847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:22.967045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:22.968267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:23.027898 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1233/2485] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:33:23.037558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:23.038983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:23.109315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a2cec-825d-439f-af66-10b83e1d5d37 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:23.113646 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:23.114437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:23.114711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:23.114979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:23.130104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:23.131538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:23.152199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1253/2486] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:23.811725 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:23.813276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:23.840524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a20418a-78f5-4a28-8da5-53945e01eae0 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:23.840524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:23.840524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:23.840524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:23.841114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] PATH_INFO: `/v3/projects/1011efd26c7d43e1b29fc653c7d3a6c7` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:23.841114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1011efd26c7d43e1b29fc653c7d3a6c7)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:23.841114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:23.997487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1011efd26c7d43e1b29fc653c7d3a6c7', 'request_id': 'req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.003774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1011efd26c7d43e1b29fc653c7d3a6c7', 'request_id': 'req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.010184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1011efd26c7d43e1b29fc653c7d3a6c7', 'request_id': 'req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.151788 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] The token cache is being invalidate because project 1011efd26c7d43e1b29fc653c7d3a6c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:24.152133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.152804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.154612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1234/2487] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:23 2022] DELETE /identity/v3/projects/1011efd26c7d43e1b29fc653c7d3a6c7 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:24.159189 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:24.173763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b957df2-2ff5-452a-b794-c8ce25c551fc service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:24.233800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:24.249365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:24.250705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:24.250705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:24.250705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] PATH_INFO: `/v3/users/12acfa06095e43419eb8ba09ce6ee3ec` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:24.252202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12acfa06095e43419eb8ba09ce6ee3ec)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:24.254686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:24.496244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12acfa06095e43419eb8ba09ce6ee3ec'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.497488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12acfa06095e43419eb8ba09ce6ee3ec'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.498283 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12acfa06095e43419eb8ba09ce6ee3ec'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.515502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12acfa06095e43419eb8ba09ce6ee3ec'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.632209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12acfa06095e43419eb8ba09ce6ee3ec'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:24.652040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1254/2488] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:24 2022] DELETE /identity/v3/users/12acfa06095e43419eb8ba09ce6ee3ec => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:25.236916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:25.238486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:25.259662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab8ac4-e1ab-4e91-b4f8-e272622856fc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:25.322948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:25.343075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:25.343739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:25.344116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:25.344622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] PATH_INFO: `/v3/projects/f291fa1c288b4ed797997f5a2b2308a8` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:25.350064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f291fa1c288b4ed797997f5a2b2308a8)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:25.352545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:25.493440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f291fa1c288b4ed797997f5a2b2308a8', 'request_id': 'req-3dc67181-7faa-4156-95b7-6795a34be3dc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.498884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f291fa1c288b4ed797997f5a2b2308a8', 'request_id': 'req-3dc67181-7faa-4156-95b7-6795a34be3dc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.504696 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f291fa1c288b4ed797997f5a2b2308a8', 'request_id': 'req-3dc67181-7faa-4156-95b7-6795a34be3dc'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.580982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] The token cache is being invalidate because project f291fa1c288b4ed797997f5a2b2308a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:25.581338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.581910 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.583389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1235/2489] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:25 2022] DELETE /identity/v3/projects/f291fa1c288b4ed797997f5a2b2308a8 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:25.587874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:25.589334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:25.602984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42d01cb-861f-4947-92f6-aef8a9c90867 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:25.661145 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:25.661792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:25.662070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:25.662340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] PATH_INFO: `/v3/users/b128fe9da597453e8921507dab605f56` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:25.663501 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b128fe9da597453e8921507dab605f56)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:25.665484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:25.892790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b128fe9da597453e8921507dab605f56'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.898036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b128fe9da597453e8921507dab605f56'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.898036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b128fe9da597453e8921507dab605f56'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:25.901391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b128fe9da597453e8921507dab605f56'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.009080 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3e44d11d-210c-4679-bab1-63e296c21ee4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b128fe9da597453e8921507dab605f56'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.015689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:26.029272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1255/2490] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:25 2022] DELETE /identity/v3/users/b128fe9da597453e8921507dab605f56 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:26.513331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:26.514861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:26.531103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc67181-7faa-4156-95b7-6795a34be3dc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:26.597008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:26.597650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:26.598661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:26.598965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] PATH_INFO: `/v3/projects/ac2577745b5f4109abc8c715e7f3ae95` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:26.600137 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2577745b5f4109abc8c715e7f3ae95)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:26.603207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:26.747423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2577745b5f4109abc8c715e7f3ae95', 'request_id': 'req-85a7641a-5fa9-424b-84a9-39970b950331'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.754119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2577745b5f4109abc8c715e7f3ae95', 'request_id': 'req-85a7641a-5fa9-424b-84a9-39970b950331'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.760163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2577745b5f4109abc8c715e7f3ae95', 'request_id': 'req-85a7641a-5fa9-424b-84a9-39970b950331'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.846431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] The token cache is being invalidate because project ac2577745b5f4109abc8c715e7f3ae95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:26.852057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.852057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-85a7641a-5fa9-424b-84a9-39970b950331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:26.852057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1236/2491] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:26 2022] DELETE /identity/v3/projects/ac2577745b5f4109abc8c715e7f3ae95 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:26.855039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257470c-f18e-4979-908f-685f2d739e43 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:26.855560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257470c-f18e-4979-908f-685f2d739e43 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:26.855674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257470c-f18e-4979-908f-685f2d739e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:26.874212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:26.874496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:26.874768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:26.875889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-b257470c-f18e-4979-908f-685f2d739e43 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:26.877125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b257470c-f18e-4979-908f-685f2d739e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:26.883722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:26.894827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:26.896209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:26.941742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1256/2492] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:26.949649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:26.950028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:26.950307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:26.954254 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1237/2493] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 10 11:33:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:26.967125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:26.968642 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:26.971629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:26.972840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:26.982697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1257/2494] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:33:26.986856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:26.995665 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac10a3f-4a17-4e66-8464-bfc062d76da9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.000997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.001589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.001862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.002149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.002758 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:33:27.003052 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.004840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.017471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1258/2495] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:33:27.021679 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:27.031978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac86e-ec80-4906-af00-38d52dfc2dcd admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.037132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.038043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.038043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.038043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.038295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.040208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.041487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc0367-658c-4947-96c9-b1e8654330bc None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.047637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.048359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.048637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.048905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.057673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.059525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.067665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1238/2496] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:33:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:33:27.176153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1259/2497] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:33:27 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:27.180445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:27.191387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca3f746-3ab8-4d7b-aaef-8d3f0599d9d4 service swift] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.250367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.250971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.251251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.251520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.252172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.254099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.383474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1239/2498] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:33:27 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:27.384781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:27.395494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dac185-91e4-4dd8-9043-ba708a17c1fd admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.399936 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.401128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.401128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.401128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.402335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.404082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.413851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1260/2499] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:27.418711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:27.428614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21009f-b6af-44c2-b0ec-28f261922978 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.433156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4384b576-be16-4a5c-a598-282d190d9abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.433843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384b576-be16-4a5c-a598-282d190d9abc admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.434120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384b576-be16-4a5c-a598-282d190d9abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.434391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4384b576-be16-4a5c-a598-282d190d9abc admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.435021 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384b576-be16-4a5c-a598-282d190d9abc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.436713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4384b576-be16-4a5c-a598-282d190d9abc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.446535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1240/2500] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:33:27.450688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:27.461604 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372354b-caeb-482e-a31e-92d913412e08 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.469451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.470164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.470544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.470793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] PATH_INFO: `/v3/projects/5f09133ba9e64aab9df9187ecd4afaee/users/51773db2ad09414d945b2a3131d7ec51/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.487367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f09133ba9e64aab9df9187ecd4afaee, user_id=51773db2ad09414d945b2a3131d7ec51, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.490190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.576874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1261/2501] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:27 2022] PUT /identity/v3/projects/5f09133ba9e64aab9df9187ecd4afaee/users/51773db2ad09414d945b2a3131d7ec51/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:27.590215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.590934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.590934 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.623242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] MFA Rules not processed for user `51773db2ad09414d945b2a3131d7ec51`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:27.624469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:27.629798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:27.698175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1241/2502] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:27.709915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:27.712425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:27.769855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b3c695-61a7-4f4a-b0c4-2a9f5db0f2af admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:27.773754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac02ae-ec48-4b3f-b986-bf391fec8d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:27.774362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac02ae-ec48-4b3f-b986-bf391fec8d4e service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:27.774643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac02ae-ec48-4b3f-b986-bf391fec8d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:27.774949 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac02ae-ec48-4b3f-b986-bf391fec8d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:27.783943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac02ae-ec48-4b3f-b986-bf391fec8d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:27.785385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac02ae-ec48-4b3f-b986-bf391fec8d4e service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:27.793997 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1262/2503] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:37.780580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=24 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:37.790000 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:41.456559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:41.516500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9f2a5-1577-42d5-a423-0ab4204f377a None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:41.520902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:41.521515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:41.521794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:41.522059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] PATH_INFO: `/v3/users/51773db2ad09414d945b2a3131d7ec51` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:41.523155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51773db2ad09414d945b2a3131d7ec51)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:41.525083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:41.760081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51773db2ad09414d945b2a3131d7ec51'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:41.760672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51773db2ad09414d945b2a3131d7ec51'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:41.762293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51773db2ad09414d945b2a3131d7ec51'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:41.769107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51773db2ad09414d945b2a3131d7ec51'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:41.859989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51773db2ad09414d945b2a3131d7ec51'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:41.878027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1242/2504] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:41 2022] DELETE /identity/v3/users/51773db2ad09414d945b2a3131d7ec51 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:41.882534 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:41.882994 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:41.883091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:41.901683 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:41.902959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:41.961057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1263/2505] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:41.971195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:41.972622 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:42.029168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0c4ebf-d801-4eac-95a6-24a864ab0d79 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:42.032560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:42.033149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:42.033425 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:42.033691 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:42.043274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:42.044713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:42.053766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1243/2506] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:42.524928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:42.526463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:42.539202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c082dbf-5770-45c0-b7d5-cb0ccbe93840 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:42.544773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:42.546371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:42.546643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:42.547078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] PATH_INFO: `/v3/projects/5f09133ba9e64aab9df9187ecd4afaee` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:42.548233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f09133ba9e64aab9df9187ecd4afaee)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:42.550154 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:42.713270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f09133ba9e64aab9df9187ecd4afaee', 'request_id': 'req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:42.719507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f09133ba9e64aab9df9187ecd4afaee', 'request_id': 'req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:42.725787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f09133ba9e64aab9df9187ecd4afaee', 'request_id': 'req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:42.826018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] The token cache is being invalidate because project 5f09133ba9e64aab9df9187ecd4afaee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:42.828395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:42.828395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:42.828623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1264/2507] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:42 2022] DELETE /identity/v3/projects/5f09133ba9e64aab9df9187ecd4afaee => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:50.949765 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:50.958394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d3a19-d9b4-4776-9a56-1e4ca50ffeda service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:50.976972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:51.006144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:51.006745 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:51.007029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:51.007297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:51.009061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:51.071577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:51.073029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:51.082950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1244/2508] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:52.822162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:55.079523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:55.080495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:55.096215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3a4d5-f270-4ca5-a215-05c2a84a014e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:55.109462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:55.109462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:55.109462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:55.109462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] PATH_INFO: `/v3/users/11be37af210c45fc9ddebf7f28b265ef` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:55.118697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11be37af210c45fc9ddebf7f28b265ef)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:55.120795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:55.434071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11be37af210c45fc9ddebf7f28b265ef'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:55.434802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11be37af210c45fc9ddebf7f28b265ef'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:55.435408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11be37af210c45fc9ddebf7f28b265ef'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:55.442471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11be37af210c45fc9ddebf7f28b265ef'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:55.522862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-2be23817-6f95-416e-90e4-ed7efde6f3b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11be37af210c45fc9ddebf7f28b265ef'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:55.540954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1265/2509] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:33:55 2022] DELETE /identity/v3/users/11be37af210c45fc9ddebf7f28b265ef => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:55.816129 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:55.826840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e9340c-734b-467a-8ede-0344b3c51c8c service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:55.887394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:55.888096 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:55.888366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:55.888632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] PATH_INFO: `/v3/projects/362ec4487088496a8198a3cf71815256` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:55.898638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=362ec4487088496a8198a3cf71815256)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:55.900656 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.031844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '362ec4487088496a8198a3cf71815256', 'request_id': 'req-56986cda-a623-4c78-b856-3daf3c44dd6f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:56.038286 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '362ec4487088496a8198a3cf71815256', 'request_id': 'req-56986cda-a623-4c78-b856-3daf3c44dd6f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:56.044400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '362ec4487088496a8198a3cf71815256', 'request_id': 'req-56986cda-a623-4c78-b856-3daf3c44dd6f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:56.046840 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:56.225300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] The token cache is being invalidate because project 362ec4487088496a8198a3cf71815256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:33:56.225560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:56.226119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-56986cda-a623-4c78-b856-3daf3c44dd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:33:56.227605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1245/2510] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:33:55 2022] DELETE /identity/v3/projects/362ec4487088496a8198a3cf71815256 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:56.232611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.233089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.233215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.252460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:56.254241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:56.312908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1266/2511] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:56.321615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.322099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.322212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.342614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:56.343801 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:56.351553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1246/2512] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:56.356532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:56.366336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6c89e-21cf-4adc-8c58-8223dc7f0a3b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:56.371546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:56.372208 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.372487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.372757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.373407 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:33:56.373733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:56.375564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.388867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1267/2513] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:33:56.399552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:56.409662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd1300-2193-4f0b-9e02-4dac27ce35bb None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:56.414345 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:56.414930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.415226 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.415475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.416196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:56.419098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.553274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1247/2514] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:33:56 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:56.557410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:56.568400 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68116b0a-708f-48ac-98dd-f4b1765b415d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:56.614193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:56.626242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:56.626836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.627111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.627382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.628020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:56.629982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.739794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1268/2515] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:33:56 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:56.743968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:56.755308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e62e7-8a36-4cf4-b7f8-bbdaaf6c2fc6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:56.759878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:56.760495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.760770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.761036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.762275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:56.763958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.773560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1248/2516] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:56.777942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:56.787671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19527a8-1090-4adb-80b6-9503e77ff995 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:56.800091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fd9cc-f772-4a85-8282-66dbd6d4053f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:56.800091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fd9cc-f772-4a85-8282-66dbd6d4053f admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.800091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fd9cc-f772-4a85-8282-66dbd6d4053f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.800091 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fd9cc-f772-4a85-8282-66dbd6d4053f admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.800486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fd9cc-f772-4a85-8282-66dbd6d4053f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:56.800486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fd9cc-f772-4a85-8282-66dbd6d4053f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.807446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1269/2517] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:56.812295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:56.822277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48141c4b-448b-4876-a775-658fd332da13 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:56.827569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:56.828238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:56.828521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:56.828798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] PATH_INFO: `/v3/projects/71c00068289f4cabbbf784c211ec4d5f/users/0645ac2b15ec4e4c93a4893c6fdc81eb/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:56.843799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71c00068289f4cabbbf784c211ec4d5f, user_id=0645ac2b15ec4e4c93a4893c6fdc81eb, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:56.846503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:56.991713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1249/2518] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:56 2022] PUT /identity/v3/projects/71c00068289f4cabbbf784c211ec4d5f/users/0645ac2b15ec4e4c93a4893c6fdc81eb/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:57.003044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.003361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.003634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.030406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] MFA Rules not processed for user `0645ac2b15ec4e4c93a4893c6fdc81eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:57.031613 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:57.094560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1270/2519] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:57.099199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:57.159851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402aa72a-4a3c-467a-aa42-520eee147a99 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:57.164454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:57.165048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.165325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.165595 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.166212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:57.168735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:57.249294 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1250/2520] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:33:57 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:57.253362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:57.264338 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb5ed5-f122-4bbb-98e4-047b7db0e28f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:57.319896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:57.320564 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.320846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.321119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.321734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:57.323691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:57.540101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1271/2521] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:33:57 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:33:57.545324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:57.555441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a1b58-4ec7-4094-8dbd-ed86be133279 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:57.560144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:57.560775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.561052 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.561338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.561952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:57.563596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:57.574447 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1251/2522] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:33:57.578490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:57.589147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5ee4e-dce7-4f9a-a14c-e4cf1e0dd1d0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:57.594553 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:57.595184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.595464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.595736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] PATH_INFO: `/v3/projects/0ded043cc3944f3db3ed55af09970a2d/users/cd7ab7c70d1140979eb00158ac586b64/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.610523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ded043cc3944f3db3ed55af09970a2d, user_id=cd7ab7c70d1140979eb00158ac586b64, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:57.613222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:57.809568 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1272/2523] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:57 2022] PUT /identity/v3/projects/0ded043cc3944f3db3ed55af09970a2d/users/cd7ab7c70d1140979eb00158ac586b64/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:33:57.813647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:57.824545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa586c9c-ad16-450d-948a-4874335499a5 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:57.881227 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400e3de5-0f33-4484-80ae-f7e182c40573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:57.881827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e3de5-0f33-4484-80ae-f7e182c40573 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.882108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e3de5-0f33-4484-80ae-f7e182c40573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.882385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400e3de5-0f33-4484-80ae-f7e182c40573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.883017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e3de5-0f33-4484-80ae-f7e182c40573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:57.884639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400e3de5-0f33-4484-80ae-f7e182c40573 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:57.890204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:57.894943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1252/2524] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:33:57.899171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:57.910073 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f867447-598e-43e6-ae83-71ddea399381 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:57.914672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:57.915317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.915593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.915863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] PATH_INFO: `/v3/projects/0ded043cc3944f3db3ed55af09970a2d/users/cd7ab7c70d1140979eb00158ac586b64/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.917955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ded043cc3944f3db3ed55af09970a2d, user_id=cd7ab7c70d1140979eb00158ac586b64, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:57.920600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:57.930627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1273/2525] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:33:57 2022] PUT /identity/v3/projects/0ded043cc3944f3db3ed55af09970a2d/users/cd7ab7c70d1140979eb00158ac586b64/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:33:57.941325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:57.941637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:57.941911 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:57.971237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] MFA Rules not processed for user `cd7ab7c70d1140979eb00158ac586b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:33:57.972503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:58.040383 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1253/2526] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:33:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:33:58.051416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:58.053119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:33:58.064378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:33:58.111063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b59e70-ab45-4214-93f5-750766f70cc6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:58.114713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:58.115311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:58.115587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:58.115856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:58.125125 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:58.126540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:58.136031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1274/2527] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:33:58.313606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:33:58.323112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad2a1-193d-469d-834f-9bef5bd87d32 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:33:58.327217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:33:58.327800 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:33:58.328108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:33:58.328378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:33:58.386765 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:33:58.388290 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:33:58.397008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1254/2528] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:33:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:34:08.137425 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=22 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:08.393107 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=62 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:52.189898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:52.191544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:52.317012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013ecd5-f3ac-4be5-bf1e-9665dc4bcc46 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:52.334529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:52.335252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:52.335579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:52.335804 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:52.338794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:52.359503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:52.362817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0605d-34f7-4098-b0b9-96ce66d04ef2 service nova] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:52.388386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1275/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:34:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:34:55.360098 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:55.426566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0def0-eda9-436b-8acc-92c38e5052f0 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:55.431054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:55.431704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:55.431983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:55.432281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] PATH_INFO: `/v3/users/0645ac2b15ec4e4c93a4893c6fdc81eb` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:55.433395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0645ac2b15ec4e4c93a4893c6fdc81eb)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:55.435430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:55.656198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0645ac2b15ec4e4c93a4893c6fdc81eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:55.657122 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0645ac2b15ec4e4c93a4893c6fdc81eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:55.657675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0645ac2b15ec4e4c93a4893c6fdc81eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:55.664440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0645ac2b15ec4e4c93a4893c6fdc81eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:55.738799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0645ac2b15ec4e4c93a4893c6fdc81eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:55.757906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1255/2530] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:34:55 2022] DELETE /identity/v3/users/0645ac2b15ec4e4c93a4893c6fdc81eb => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:34:55.762408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:55.762897 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:55.763443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:55.782331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:34:55.783524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:55.840514 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1276/2531] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:34:55.850814 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:55.852297 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:55.909551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ced2cf-54ac-4f2e-8fcc-b7aeb4d50f60 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:55.913112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:55.913713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:55.913994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:55.914269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:55.923627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:55.925075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:55.934363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1256/2532] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:34:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:34:56.369128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:56.370502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:56.382931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6e247b-a036-49ed-99a2-7c261da466bc None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:56.387790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:56.388464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:56.388745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:56.389060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] PATH_INFO: `/v3/projects/71c00068289f4cabbbf784c211ec4d5f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:56.390302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71c00068289f4cabbbf784c211ec4d5f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:56.392360 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:56.522729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71c00068289f4cabbbf784c211ec4d5f', 'request_id': 'req-037cc0d8-b541-42a3-89bd-d229dc01fdf0'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.528656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71c00068289f4cabbbf784c211ec4d5f', 'request_id': 'req-037cc0d8-b541-42a3-89bd-d229dc01fdf0'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.535535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71c00068289f4cabbbf784c211ec4d5f', 'request_id': 'req-037cc0d8-b541-42a3-89bd-d229dc01fdf0'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.635851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] The token cache is being invalidate because project 71c00068289f4cabbbf784c211ec4d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:34:56.636169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.636793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.638271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1277/2533] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:34:56 2022] DELETE /identity/v3/projects/71c00068289f4cabbbf784c211ec4d5f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:34:56.642677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:56.652423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6124d-3fca-48a6-b780-b5e8f688682f service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:56.671232 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:56.709857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:56.710480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:56.710757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:56.711025 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] PATH_INFO: `/v3/users/cd7ab7c70d1140979eb00158ac586b64` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:56.712474 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7ab7c70d1140979eb00158ac586b64)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:56.714435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:56.992742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7ab7c70d1140979eb00158ac586b64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.993406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7ab7c70d1140979eb00158ac586b64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:56.993953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7ab7c70d1140979eb00158ac586b64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.000694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7ab7c70d1140979eb00158ac586b64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.092925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7ab7c70d1140979eb00158ac586b64'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.110494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1257/2534] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:34:56 2022] DELETE /identity/v3/users/cd7ab7c70d1140979eb00158ac586b64 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:34:57.553922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:57.555617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:57.568475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:57.569944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:57.581601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cc0d8-b541-42a3-89bd-d229dc01fdf0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:57.588174 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:57.638140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ac236-bc84-4cfd-8c21-02b8ed604ee3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:57.641664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:57.642295 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:57.642628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:57.642774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:57.643030 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] PATH_INFO: `/v3/projects/0ded043cc3944f3db3ed55af09970a2d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:57.643228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:57.643677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:57.643774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:57.644365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ded043cc3944f3db3ed55af09970a2d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:57.645706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:57.646416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:57.658494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:57.659930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3df88-aadb-4808-9ea3-ed675e0ba894 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:57.670150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1258/2535] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:34:57.831566 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ded043cc3944f3db3ed55af09970a2d', 'request_id': 'req-aad86ef2-038f-4239-a44b-67c662f19ed5'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.837666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ded043cc3944f3db3ed55af09970a2d', 'request_id': 'req-aad86ef2-038f-4239-a44b-67c662f19ed5'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.844833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ded043cc3944f3db3ed55af09970a2d', 'request_id': 'req-aad86ef2-038f-4239-a44b-67c662f19ed5'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.957621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] The token cache is being invalidate because project 0ded043cc3944f3db3ed55af09970a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:34:57.957889 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.958582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-aad86ef2-038f-4239-a44b-67c662f19ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:34:57.960435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1278/2536] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:34:57 2022] DELETE /identity/v3/projects/0ded043cc3944f3db3ed55af09970a2d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:34:57.965545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:57.976819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:57.976819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:57.987366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:34:57.988706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:58.029337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:58.048836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1259/2537] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:34:58.056662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.056968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.057484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.105098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:34:58.105098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:58.105098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1279/2538] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:34:58.107983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.118340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53393d-2301-4131-ba67-8ad5f0857413 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.122907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.123503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.123779 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.124072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.124722 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:34:58.125023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.126796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.139220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1260/2539] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:34:58.154095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.163984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d1a83-aea4-4a4b-9aa0-0dbb72ec731b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.168673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.169278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.169551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.169814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.170442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.173261 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.287384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1280/2540] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:34:58 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:34:58.288623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.299503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b44c34-9ce8-4596-92e1-75ffa4d7db11 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.358111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.358730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.359009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.359279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.360468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.362546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.463125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1261/2541] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:34:58 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:34:58.464292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.475887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b85c2-f4c8-44a8-953d-376a52736c59 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.480961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.484421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.484421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.484421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.484421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.484421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.495794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1281/2542] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:34:58.500106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.511516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc57b8be-291c-4775-86bf-be2d7e304aa0 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.516626 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f2ee2-25b0-4574-8d17-e6667e00174a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.522444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f2ee2-25b0-4574-8d17-e6667e00174a admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.522444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f2ee2-25b0-4574-8d17-e6667e00174a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.522444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f2ee2-25b0-4574-8d17-e6667e00174a admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.522444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f2ee2-25b0-4574-8d17-e6667e00174a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.522444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f2ee2-25b0-4574-8d17-e6667e00174a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.529633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1262/2543] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:34:58.534399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.544209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b388f9e-aac5-4653-9fc4-8fa5528270c6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.549421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.550070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.555432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.555432 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] PATH_INFO: `/v3/projects/cc0b76e1cfba421faac0a6b23ab21565/users/78ea821a59534bfd918d1d9292b84b1f/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.565075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc0b76e1cfba421faac0a6b23ab21565, user_id=78ea821a59534bfd918d1d9292b84b1f, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.567702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.673493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1282/2544] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:34:58 2022] PUT /identity/v3/projects/cc0b76e1cfba421faac0a6b23ab21565/users/78ea821a59534bfd918d1d9292b84b1f/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:34:58.674496 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.675569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.676097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.712713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] MFA Rules not processed for user `78ea821a59534bfd918d1d9292b84b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:34:58.714204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:58.793503 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1263/2545] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:34:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:34:58.798156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.842709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:58.864987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd0d83-3581-4440-863b-17c17b9ab92a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:58.869992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:58.870631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:58.870914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:58.871188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:58.871816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:58.873779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:58.971027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1283/2546] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:34:58 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:34:58.974962 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:58.986083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde2331-9ade-450d-a340-11aae1053f80 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:59.024920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:34:59.043427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:59.044145 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.044494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.044793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.045459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:59.047385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:59.147551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1264/2547] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:34:58 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:34:59.150427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:59.161769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff0a00-907d-4f9d-9877-25e98775049d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:59.167319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:59.168132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.168416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.168692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.169332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:59.170965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:59.181813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1284/2548] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:34:59.185990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:59.196244 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f38e4-6266-4e05-bca2-fc09f69be29f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:59.204281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:59.206321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.206321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.206321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] PATH_INFO: `/v3/projects/5de57a9a5fd24bc4ba4403127fc63023/users/f7fdbe5b3d1540f48932575278d1f0f0/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.221850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de57a9a5fd24bc4ba4403127fc63023, user_id=f7fdbe5b3d1540f48932575278d1f0f0, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:59.224772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:59.323513 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1265/2549] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:34:59 2022] PUT /identity/v3/projects/5de57a9a5fd24bc4ba4403127fc63023/users/f7fdbe5b3d1540f48932575278d1f0f0/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:34:59.327739 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:59.339367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf0506-b988-43d1-826b-c3099739d983 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:59.411517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2caa9c-69ad-44e8-97dc-dc609d454763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:59.412184 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2caa9c-69ad-44e8-97dc-dc609d454763 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.412468 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2caa9c-69ad-44e8-97dc-dc609d454763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.412740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2caa9c-69ad-44e8-97dc-dc609d454763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.413387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2caa9c-69ad-44e8-97dc-dc609d454763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:59.415005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2caa9c-69ad-44e8-97dc-dc609d454763 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:59.426879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1285/2550] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:34:59.431059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:59.443258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9c01fa-d47a-431b-8c96-e08f6a3b6e21 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:59.447929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:59.448724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.449465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.449465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] PATH_INFO: `/v3/projects/5de57a9a5fd24bc4ba4403127fc63023/users/f7fdbe5b3d1540f48932575278d1f0f0/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.451480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de57a9a5fd24bc4ba4403127fc63023, user_id=f7fdbe5b3d1540f48932575278d1f0f0, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:59.454459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:59.467229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1266/2551] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:34:59 2022] PUT /identity/v3/projects/5de57a9a5fd24bc4ba4403127fc63023/users/f7fdbe5b3d1540f48932575278d1f0f0/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:34:59.478640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.478827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.479099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.510563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] MFA Rules not processed for user `f7fdbe5b3d1540f48932575278d1f0f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:34:59.511890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:59.602159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1286/2552] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:34:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:34:59.610457 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:34:59.612377 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:34:59.676197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bdeeb-8593-4506-91a2-71e6dea3a2fe admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:34:59.680123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5823303-5fc5-426b-aba8-44b3c595767a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:34:59.680737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5823303-5fc5-426b-aba8-44b3c595767a service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:34:59.681038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5823303-5fc5-426b-aba8-44b3c595767a service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:34:59.681311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5823303-5fc5-426b-aba8-44b3c595767a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:34:59.690645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5823303-5fc5-426b-aba8-44b3c595767a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:34:59.692177 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5823303-5fc5-426b-aba8-44b3c595767a service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:34:59.701929 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1267/2553] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:34:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:00.761696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:00.795991 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:00.824888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8854c6-06da-4885-b0e3-904be9c79328 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:00.830089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:00.830727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:00.831004 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:00.831271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] PATH_INFO: `/v3/users/78ea821a59534bfd918d1d9292b84b1f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:00.832420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ea821a59534bfd918d1d9292b84b1f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:00.834549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:01.067105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ea821a59534bfd918d1d9292b84b1f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.067856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ea821a59534bfd918d1d9292b84b1f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.068530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ea821a59534bfd918d1d9292b84b1f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.076820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ea821a59534bfd918d1d9292b84b1f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.196126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ea821a59534bfd918d1d9292b84b1f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.220109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1287/2554] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:00 2022] DELETE /identity/v3/users/78ea821a59534bfd918d1d9292b84b1f => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:01.225943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:01.226438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:01.226927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:01.257531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:01.259138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:01.336813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1268/2555] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:01.346873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:01.348386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:01.411485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa96af0-0430-4737-9444-1b161baf8592 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:01.415121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:01.415726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:01.416028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:01.416301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:01.426107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:01.427554 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:01.437225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1288/2556] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:01.552168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:01.553977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:01.568574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c83ed-c5fa-4011-8a77-593d04343ba7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:01.573168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:01.573780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:01.574054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:01.574320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] PATH_INFO: `/v3/users/87adb637dd814cf8be9ff6c2a2bc26ea` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:01.575419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] RBAC: Authorizing `identity:delete_user(user_id=87adb637dd814cf8be9ff6c2a2bc26ea)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:01.577350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:01.769084 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:01.782547 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87adb637dd814cf8be9ff6c2a2bc26ea'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.783215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87adb637dd814cf8be9ff6c2a2bc26ea'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.783833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87adb637dd814cf8be9ff6c2a2bc26ea'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.790585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87adb637dd814cf8be9ff6c2a2bc26ea'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.860538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:01.862027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:01.878923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac247df-24eb-454f-b8e1-d045ab5bcd3f service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:01.883792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:01.884491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:01.884773 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:01.885051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] PATH_INFO: `/v3/projects/cc0b76e1cfba421faac0a6b23ab21565` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:01.886300 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc0b76e1cfba421faac0a6b23ab21565)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:01.888326 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:01.899805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87adb637dd814cf8be9ff6c2a2bc26ea'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:01.918895 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1269/2557] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:01 2022] DELETE /identity/v3/users/87adb637dd814cf8be9ff6c2a2bc26ea => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:02.037341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:02.038730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc0b76e1cfba421faac0a6b23ab21565', 'request_id': 'req-eb268516-a946-4f8b-a589-628b912d6190'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.044853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc0b76e1cfba421faac0a6b23ab21565', 'request_id': 'req-eb268516-a946-4f8b-a589-628b912d6190'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.052214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc0b76e1cfba421faac0a6b23ab21565', 'request_id': 'req-eb268516-a946-4f8b-a589-628b912d6190'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.121528 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] The token cache is being invalidate because project cc0b76e1cfba421faac0a6b23ab21565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:02.121752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.122349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.123855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1289/2558] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:01 2022] DELETE /identity/v3/projects/cc0b76e1cfba421faac0a6b23ab21565 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:02.128634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:02.130024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:02.152093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:02.157123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:02.206832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340adbf3-8d3a-431f-a1de-0411603cd14d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:02.227943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb268516-a946-4f8b-a589-628b912d6190 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:02.232687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:02.233313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.233593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.233864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] PATH_INFO: `/v3/projects/282c9e4b65d64085a0ba1a3d356f59fa` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.235853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=282c9e4b65d64085a0ba1a3d356f59fa)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:02.237669 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:02.266036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:02.266664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.266941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.267213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] PATH_INFO: `/v3/users/f7fdbe5b3d1540f48932575278d1f0f0` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.268374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fdbe5b3d1540f48932575278d1f0f0)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:02.270270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:02.356402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '282c9e4b65d64085a0ba1a3d356f59fa', 'request_id': 'req-6b68d6c4-2620-4512-b527-37347d9cab34'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.362397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '282c9e4b65d64085a0ba1a3d356f59fa', 'request_id': 'req-6b68d6c4-2620-4512-b527-37347d9cab34'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.368786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '282c9e4b65d64085a0ba1a3d356f59fa', 'request_id': 'req-6b68d6c4-2620-4512-b527-37347d9cab34'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.498937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] The token cache is being invalidate because project 282c9e4b65d64085a0ba1a3d356f59fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:02.499254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.499968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6b68d6c4-2620-4512-b527-37347d9cab34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.501840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1290/2559] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:02 2022] DELETE /identity/v3/projects/282c9e4b65d64085a0ba1a3d356f59fa => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:02.506386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282dd20-b5d1-480b-b915-317383124bdb None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.506664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282dd20-b5d1-480b-b915-317383124bdb None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.511985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282dd20-b5d1-480b-b915-317383124bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.523170 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fdbe5b3d1540f48932575278d1f0f0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.523830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fdbe5b3d1540f48932575278d1f0f0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.524536 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fdbe5b3d1540f48932575278d1f0f0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.528985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9282dd20-b5d1-480b-b915-317383124bdb None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:02.530115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9282dd20-b5d1-480b-b915-317383124bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:02.534410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fdbe5b3d1540f48932575278d1f0f0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.536217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:02.595788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1291/2560] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:02.600575 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.600855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.601123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.616059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:02.621272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:02.622743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:02.631178 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1292/2561] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:02.635160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:02.644323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a93a8-c6c0-46c3-bae5-90020c761ebf None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:02.647268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fdbe5b3d1540f48932575278d1f0f0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:02.649248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:02.649838 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.650118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.650387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.651051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:35:02.651347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:02.653211 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:02.665522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1293/2562] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:35:02.667324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1270/2563] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:02 2022] DELETE /identity/v3/users/f7fdbe5b3d1540f48932575278d1f0f0 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:02.669827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:02.681005 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb8885-97e8-43ef-809f-654fd10c9cd3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:02.738648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:02.739186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.739476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.739747 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.740408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:02.742294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:02.837995 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1294/2564] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:02.842260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:02.853167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71a493-c214-4ff9-897f-cb24728b7c52 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:02.918417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:02.918417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:02.918417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:02.918417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:02.919123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:02.919123 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.059566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1271/2565] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:02 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:03.064793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.075495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e5991-26ac-48d9-83aa-d6ba25454e20 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.081292 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.081886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.082159 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.082426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.083055 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.084666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.096435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1295/2566] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:03.100405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.111090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a97713-da33-4fb1-8d2d-67f95e76e4c6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.115671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.116311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.116593 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.116867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.117500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.119071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.129117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1272/2567] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:03.133228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.143524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c0b9a-9f71-4ff5-aeb9-e6e40006b457 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.148132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5cf77b-4309-4943-b11d-68a5ae82d792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.148778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf77b-4309-4943-b11d-68a5ae82d792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.149057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf77b-4309-4943-b11d-68a5ae82d792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.149331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5cf77b-4309-4943-b11d-68a5ae82d792 admin admin] PATH_INFO: `/v3/projects/9c1cb085c9914377898584966c934d33/users/7b0e07f56c484aca8c8b9cb900b7a3ec/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.163856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cf77b-4309-4943-b11d-68a5ae82d792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1cb085c9914377898584966c934d33, user_id=7b0e07f56c484aca8c8b9cb900b7a3ec, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.166555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5cf77b-4309-4943-b11d-68a5ae82d792 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.330954 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.336600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:03.346617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e9072-7477-4849-b8af-eb9d76389f80 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.351387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.352046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.352333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.352606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] PATH_INFO: `/v3/projects/5de57a9a5fd24bc4ba4403127fc63023` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.353717 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de57a9a5fd24bc4ba4403127fc63023)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.355641 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.378113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1296/2568] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:03 2022] PUT /identity/v3/projects/9c1cb085c9914377898584966c934d33/users/7b0e07f56c484aca8c8b9cb900b7a3ec/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:03.389618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.389903 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.390177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.417474 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] MFA Rules not processed for user `7b0e07f56c484aca8c8b9cb900b7a3ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:03.418668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:03.480946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1297/2569] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:03.485590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.515480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:03.550404 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e33893-1d0c-4574-a25a-0b3c13a81df9 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.555479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.556123 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.556405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.556677 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.557322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.559212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.625278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de57a9a5fd24bc4ba4403127fc63023', 'request_id': 'req-ae4128b6-c24f-4aaa-af15-5ba72af3c387'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:03.632141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de57a9a5fd24bc4ba4403127fc63023', 'request_id': 'req-ae4128b6-c24f-4aaa-af15-5ba72af3c387'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:03.637708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de57a9a5fd24bc4ba4403127fc63023', 'request_id': 'req-ae4128b6-c24f-4aaa-af15-5ba72af3c387'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:03.757666 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1298/2570] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:03 2022] POST /identity/v3/projects => generated 375 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:03.770648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.808545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] The token cache is being invalidate because project 5de57a9a5fd24bc4ba4403127fc63023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:03.808839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:03.809482 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ae4128b6-c24f-4aaa-af15-5ba72af3c387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:03.810992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1273/2571] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:03 2022] DELETE /identity/v3/projects/5de57a9a5fd24bc4ba4403127fc63023 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:03.816344 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea96764-40cb-455b-bc88-81194bf97c6b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.816764 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea96764-40cb-455b-bc88-81194bf97c6b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.817117 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea96764-40cb-455b-bc88-81194bf97c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.823725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef960-047b-41e0-8315-0f4a93c0d653 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.831001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.831693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.832111 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.832384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.833109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.835017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.838007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-fea96764-40cb-455b-bc88-81194bf97c6b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:03.839221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fea96764-40cb-455b-bc88-81194bf97c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:03.847387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1274/2572] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:03.855082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.855388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.855648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.874452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:03.876404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:03.884465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1275/2573] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:03.888524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.898287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a825dd-7524-490a-be84-2637657e33c6 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.902873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.903435 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.903711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.903982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.904647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:35:03.904946 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.906554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.918037 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1276/2574] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:35:03.922701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.932729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1299/2575] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:03 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:03.932905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93762da9-1c40-429d-9b52-922254ec7ad7 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.933821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.935435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:03.937231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.938394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.938789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.939082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.939704 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.941634 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.947058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:INSERT=5,keystone:DELETE=10,keystone:SELECT=85 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:03.948435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd7684-bc40-41df-a445-2e55e776a957 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.954026 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.954601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.954886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.955158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:03.955783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:03.957440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:03.967894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1300/2576] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:03.971950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:03.981331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee0c57a-69aa-41af-a513-48833ad6ee33 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:03.986285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:03.986947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:03.987226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:03.987551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] PATH_INFO: `/v3/projects/c29fc03a820d453eab6ec70979530bfc/users/76560436e33842aaba8206737b917737/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.002711 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29fc03a820d453eab6ec70979530bfc, user_id=76560436e33842aaba8206737b917737, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.005387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.057635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1277/2577] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:03 2022] POST /identity/v3/projects => generated 361 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:04.058662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.099927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1301/2578] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:03 2022] PUT /identity/v3/projects/c29fc03a820d453eab6ec70979530bfc/users/76560436e33842aaba8206737b917737/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:04.104081 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.121596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451aa8e6-ac47-4a58-a377-84f7b6f8c178 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.166658 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01411f5b-a223-41fd-b9b1-6c3e7c7c29c5 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.171464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.172753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.173028 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.173291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.173914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.175535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.178404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.179133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.179416 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.179687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.180348 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.182270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.185714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1302/2579] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:35:04.191065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.196003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:04.201110 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd585e-4d07-4a31-a341-2c858f2da968 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.206312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efdda9-2882-4bd9-89d0-3f98dda0aa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.206955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efdda9-2882-4bd9-89d0-3f98dda0aa7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.207222 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efdda9-2882-4bd9-89d0-3f98dda0aa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.223160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efdda9-2882-4bd9-89d0-3f98dda0aa7c admin admin] PATH_INFO: `/v3/projects/c29fc03a820d453eab6ec70979530bfc/users/76560436e33842aaba8206737b917737/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.223160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efdda9-2882-4bd9-89d0-3f98dda0aa7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c29fc03a820d453eab6ec70979530bfc, user_id=76560436e33842aaba8206737b917737, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.223160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1efdda9-2882-4bd9-89d0-3f98dda0aa7c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.223160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1303/2580] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:04 2022] PUT /identity/v3/projects/c29fc03a820d453eab6ec70979530bfc/users/76560436e33842aaba8206737b917737/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:04.232476 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.232757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.233029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.260558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] MFA Rules not processed for user `76560436e33842aaba8206737b917737`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:04.261763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:04.329077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1304/2581] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:04 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:04.343228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.345289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:04.387591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1278/2582] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:04 2022] POST /identity/v3/users => generated 281 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:04.405298 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85db628-6cf6-42e8-b335-c691ce29aa15 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.407036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.409199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.409806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.410108 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.410378 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.412216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:04.417292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99aae42-2f54-4860-970e-a65f22b7bc3c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.475319 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.475495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.475930 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.476237 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.476630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.477060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.477273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.478923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.489795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1305/2583] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:35:04.493083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1279/2584] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:35:04.497757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.511459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf41e2-58db-4672-9567-535c503a5d82 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.517085 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e4de6-c78e-475c-a206-a7cce7d4a6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.517695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4de6-c78e-475c-a206-a7cce7d4a6fe admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.517976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4de6-c78e-475c-a206-a7cce7d4a6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.518241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e4de6-c78e-475c-a206-a7cce7d4a6fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.518874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4de6-c78e-475c-a206-a7cce7d4a6fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.520490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e4de6-c78e-475c-a206-a7cce7d4a6fe admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.534794 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1306/2585] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:04.535813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.545830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32807c-a745-4817-8f53-8360354dcd21 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.550991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.551648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.551931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.552280 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] PATH_INFO: `/v3/projects/34b1ff52c9cb4a29b04384ba34b7b750/users/ae73576a4dc84b97987242708499131a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.567774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b1ff52c9cb4a29b04384ba34b7b750, user_id=ae73576a4dc84b97987242708499131a, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.570648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.648646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1280/2586] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:04 2022] PUT /identity/v3/projects/34b1ff52c9cb4a29b04384ba34b7b750/users/ae73576a4dc84b97987242708499131a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:04.664837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.665057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.665331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.692853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] MFA Rules not processed for user `ae73576a4dc84b97987242708499131a`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:04.694060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:04.759090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1307/2587] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:04.763648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.825514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f9961-976d-427e-961d-e03839bda2ae admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.830476 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:04.831083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:04.831357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:04.831624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:04.832352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:04.834248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:04.934648 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1281/2588] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:04 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:04.939193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:04.949649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b273487-691d-4506-958c-e3a57ca48396 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:04.985708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:05.008392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.009044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.009478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.009752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.010365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.012362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.059917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:05.113822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1308/2589] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:04 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:05.120371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:05.127059 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c274fa9-fbaa-4c4a-a77a-b1e75e6467ee admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:05.132285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.132889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.133164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.133433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.134068 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.135690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.148011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1282/2590] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:05.152362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:05.163845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee5f08-8838-4b26-bbb0-d04e0ee954dc admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:05.168960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.172153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.172153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.172153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] PATH_INFO: `/v3/projects/8e723f1a5f8643ae8b95c014de97667d/users/0bf99fb2ec074601a2194e25e7424db2/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.186433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e723f1a5f8643ae8b95c014de97667d, user_id=0bf99fb2ec074601a2194e25e7424db2, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.189136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.236275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1309/2591] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:05 2022] PUT /identity/v3/projects/8e723f1a5f8643ae8b95c014de97667d/users/0bf99fb2ec074601a2194e25e7424db2/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:05.240527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:05.251141 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e745bd-631d-4de3-8ebb-b1948d439eae admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:05.308418 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518b4a3-5c52-44d6-b1fc-59814962b82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.309040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b4a3-5c52-44d6-b1fc-59814962b82f admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.309311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b4a3-5c52-44d6-b1fc-59814962b82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.309575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518b4a3-5c52-44d6-b1fc-59814962b82f admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.310209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b4a3-5c52-44d6-b1fc-59814962b82f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.311937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518b4a3-5c52-44d6-b1fc-59814962b82f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.322007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1283/2592] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:05.326320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:05.338090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fa7e05-44a6-45a6-852c-0dc1d66692a1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:05.342886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.343531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.343806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.344802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] PATH_INFO: `/v3/projects/8e723f1a5f8643ae8b95c014de97667d/users/0bf99fb2ec074601a2194e25e7424db2/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.346703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e723f1a5f8643ae8b95c014de97667d, user_id=0bf99fb2ec074601a2194e25e7424db2, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.349441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.359512 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1310/2593] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:05 2022] PUT /identity/v3/projects/8e723f1a5f8643ae8b95c014de97667d/users/0bf99fb2ec074601a2194e25e7424db2/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:05.374175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.374467 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.374770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.404943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] MFA Rules not processed for user `0bf99fb2ec074601a2194e25e7424db2`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:05.406175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:05.463410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:05.481931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1284/2594] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:05.494441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:05.496142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:05.554160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7da7e3-8fbf-4397-b0e1-28a4101631c1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:05.558311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.562753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.562753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.562753 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.569296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.570722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.580812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1311/2595] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:05.787887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:05.797388 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc3a8b-b122-4b0f-abed-abd69c234674 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:05.800906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:05.801493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:05.801771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:05.802044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:05.857216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:05.858639 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:05.867399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1285/2596] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:06.569636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:06.574122 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:06.579490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252b888-7fe6-463e-8a68-9f32e8b1dbcf service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:06.583420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:06.584193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:06.584513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:06.584785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:06.586817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:06.651735 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:06.653410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:06.662898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1312/2597] 10.222.0.13 () {60 vars in 1396 bytes} [Wed Aug 10 11:35:06 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:11.819927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:11.828449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa7513b-9252-4e33-9c0a-7cfac4fc8336 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:11.832742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12504504-8206-4c46-8fe8-1d5ed429d604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:11.833323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12504504-8206-4c46-8fe8-1d5ed429d604 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:11.833609 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12504504-8206-4c46-8fe8-1d5ed429d604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:11.833870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12504504-8206-4c46-8fe8-1d5ed429d604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:11.894752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12504504-8206-4c46-8fe8-1d5ed429d604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:11.896253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12504504-8206-4c46-8fe8-1d5ed429d604 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:11.905525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1286/2598] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:16.676132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=27 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:21.901516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=62 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:31.675093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:31.742712 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09189dee-81ce-4822-b969-aa0fe21b226b service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:31.747329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:31.747975 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:31.748273 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:31.748538 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] PATH_INFO: `/v3/users/ae73576a4dc84b97987242708499131a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:31.749804 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae73576a4dc84b97987242708499131a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:31.751731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:31.935866 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae73576a4dc84b97987242708499131a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:31.936630 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae73576a4dc84b97987242708499131a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:31.937268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae73576a4dc84b97987242708499131a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:31.944862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae73576a4dc84b97987242708499131a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:32.030094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae73576a4dc84b97987242708499131a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:32.048760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1313/2599] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:31 2022] DELETE /identity/v3/users/ae73576a4dc84b97987242708499131a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:32.054258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:32.054737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:32.054858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:32.074038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:32.075269 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:32.135742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1287/2600] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:32.146181 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:32.147562 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:32.205165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82167879-2a05-4cf4-b2c5-500d0ba93a49 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:32.209601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:32.210210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:32.210484 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:32.210757 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:32.220576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:32.222025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:32.233039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1314/2601] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:32.764266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:32.765849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:32.780079 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238e472c-05f0-4cde-9909-3c425659cbc8 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:32.784940 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:32.785578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:32.786711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:32.787233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] PATH_INFO: `/v3/projects/34b1ff52c9cb4a29b04384ba34b7b750` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:32.788428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b1ff52c9cb4a29b04384ba34b7b750)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:32.790329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:32.917020 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b1ff52c9cb4a29b04384ba34b7b750', 'request_id': 'req-96b9750b-f6ab-48b1-8ef9-73a0d5721763'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:32.932410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b1ff52c9cb4a29b04384ba34b7b750', 'request_id': 'req-96b9750b-f6ab-48b1-8ef9-73a0d5721763'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:32.940131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b1ff52c9cb4a29b04384ba34b7b750', 'request_id': 'req-96b9750b-f6ab-48b1-8ef9-73a0d5721763'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.018553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] The token cache is being invalidate because project 34b1ff52c9cb4a29b04384ba34b7b750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:33.018846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.019483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.021046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1288/2602] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:32 2022] DELETE /identity/v3/projects/34b1ff52c9cb4a29b04384ba34b7b750 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:33.025023 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:33.034782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef02de-9ca5-4a10-b881-74bf13c10310 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:33.055367 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:33.096716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:33.097373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:33.097643 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:33.097908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] PATH_INFO: `/v3/users/0bf99fb2ec074601a2194e25e7424db2` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:33.100124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bf99fb2ec074601a2194e25e7424db2)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:33.102020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:33.261821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bf99fb2ec074601a2194e25e7424db2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.262504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bf99fb2ec074601a2194e25e7424db2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.263136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bf99fb2ec074601a2194e25e7424db2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.269940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bf99fb2ec074601a2194e25e7424db2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.321720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ccf13d7-b101-46ce-adb6-710615bde0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bf99fb2ec074601a2194e25e7424db2'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.342424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1315/2603] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:33 2022] DELETE /identity/v3/users/0bf99fb2ec074601a2194e25e7424db2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:33.760742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:33.762283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:33.777402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9750b-f6ab-48b1-8ef9-73a0d5721763 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:33.837299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:33.838309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:33.838594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:33.838868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] PATH_INFO: `/v3/projects/8e723f1a5f8643ae8b95c014de97667d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:33.840017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e723f1a5f8643ae8b95c014de97667d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:33.842001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:33.934580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:33.953516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e723f1a5f8643ae8b95c014de97667d', 'request_id': 'req-9cb1560c-fe43-44d9-9034-7ee698e5bd78'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.960489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e723f1a5f8643ae8b95c014de97667d', 'request_id': 'req-9cb1560c-fe43-44d9-9034-7ee698e5bd78'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:33.966184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e723f1a5f8643ae8b95c014de97667d', 'request_id': 'req-9cb1560c-fe43-44d9-9034-7ee698e5bd78'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:34.035400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] The token cache is being invalidate because project 8e723f1a5f8643ae8b95c014de97667d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:34.035589 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:34.036440 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-9cb1560c-fe43-44d9-9034-7ee698e5bd78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:34.037927 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1289/2604] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:33 2022] DELETE /identity/v3/projects/8e723f1a5f8643ae8b95c014de97667d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:34.042979 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.043466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.043574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.063937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:34.065238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:34.124981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1316/2605] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:34.134637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.134923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.135224 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.154585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:34.155780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:34.163686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1290/2606] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:34.167883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.178078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91152259-f91a-4e97-bb61-9d070a31c5db None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.182656 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.183250 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.183530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.183802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.184481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:35:34.184782 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.186368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.198217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1317/2607] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:35:34.202320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.211545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcc498-b699-47de-b543-582ffe7f0dba None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.216410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.216990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.217267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.217537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.218157 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.220046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.306174 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1291/2608] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/projects => generated 369 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:34.310610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.321674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac5f58f-e56f-47dd-bdeb-228807af4bdc admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.347861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:34.378875 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.385761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.385761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.385761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.385761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.385761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.476513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1318/2609] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:34.480591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.491429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8467-7a3c-47e8-8025-2d6250d6501e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.496040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.496632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.496913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.497184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.497826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.499410 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.509804 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1292/2610] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:34.513743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.524104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cba2c-c1f9-462c-ac48-aa319a83074b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.528726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340109f-4e86-4064-b48d-e6758b4828dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.529303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340109f-4e86-4064-b48d-e6758b4828dc admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.529577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340109f-4e86-4064-b48d-e6758b4828dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.529844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340109f-4e86-4064-b48d-e6758b4828dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.530467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340109f-4e86-4064-b48d-e6758b4828dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.532116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340109f-4e86-4064-b48d-e6758b4828dc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.542661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1319/2611] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:34.546798 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.557629 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc3a58-afc6-433e-a806-f8beb70950df admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.562138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.562777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.563056 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.563330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] PATH_INFO: `/v3/projects/a86b38f288f24ef1ac020b2327301845/users/0b64c8cb38e64a38a70402c92a9bd6ed/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.577790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a86b38f288f24ef1ac020b2327301845, user_id=0b64c8cb38e64a38a70402c92a9bd6ed, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.580465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.641133 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1293/2612] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:34 2022] PUT /identity/v3/projects/a86b38f288f24ef1ac020b2327301845/users/0b64c8cb38e64a38a70402c92a9bd6ed/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:34.651330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.651619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.651930 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.681849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] MFA Rules not processed for user `0b64c8cb38e64a38a70402c92a9bd6ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:34.683216 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:34.746793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1320/2613] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:34.751260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.819180 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8575bdb8-4f04-403b-91f5-05942c5191a3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.824014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.824666 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.824943 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.825206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.825828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.827702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:34.912777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1294/2614] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/projects => generated 369 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:34.916617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:34.927053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655cc87-98a5-4ee6-a77b-68c92c157373 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:34.982941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:34.983550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:34.983825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:34.984121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:34.984742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:34.986621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:35.082226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1321/2615] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:34 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:35.086598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:35.097429 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd70fb1-73b5-49e7-9ef5-5e2d42fa013a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:35.102767 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:35.103399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:35.103676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:35.103947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:35.104650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:35.106270 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:35.116904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1295/2616] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:35.121043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:35.131541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53bbb8-eabb-4fb6-aec0-ba8e237cc5b3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:35.136192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:35.136832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:35.137109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:35.137380 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] PATH_INFO: `/v3/projects/b6ba13b5f2464c6cb47821db7499a7e4/users/186727cf1fbd4869996fe75c98f60cc1/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:35.152851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ba13b5f2464c6cb47821db7499a7e4, user_id=186727cf1fbd4869996fe75c98f60cc1, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:35.155469 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:35.214493 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1322/2617] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:35 2022] PUT /identity/v3/projects/b6ba13b5f2464c6cb47821db7499a7e4/users/186727cf1fbd4869996fe75c98f60cc1/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:35.218643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:35.229112 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cc489-2aef-42da-81e2-729bdde8100e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:35.249735 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:35.287229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76830ed-1743-4e05-bccc-d2e3b1a94e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:35.287944 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76830ed-1743-4e05-bccc-d2e3b1a94e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:35.289358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76830ed-1743-4e05-bccc-d2e3b1a94e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:35.289358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76830ed-1743-4e05-bccc-d2e3b1a94e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:35.289358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76830ed-1743-4e05-bccc-d2e3b1a94e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:35.290813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76830ed-1743-4e05-bccc-d2e3b1a94e3c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:35.301554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1296/2618] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:35.305169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:35.310617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:35.315724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70e446f-2ca4-4730-a3b5-f7a0c11c8ad2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:35.321006 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:35.321659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:35.322135 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:35.322426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] PATH_INFO: `/v3/projects/b6ba13b5f2464c6cb47821db7499a7e4/users/186727cf1fbd4869996fe75c98f60cc1/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:35.324361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ba13b5f2464c6cb47821db7499a7e4, user_id=186727cf1fbd4869996fe75c98f60cc1, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:35.326960 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:35.338024 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1323/2619] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:35 2022] PUT /identity/v3/projects/b6ba13b5f2464c6cb47821db7499a7e4/users/186727cf1fbd4869996fe75c98f60cc1/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:35.348127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:35.348473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:35.348757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:35.377494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] MFA Rules not processed for user `186727cf1fbd4869996fe75c98f60cc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:35.381341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:35.452138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1297/2620] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:35.464430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:35.467981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:35.525788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8325eaf-aa5b-4d6d-bab3-087e0d2ac7eb admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:35.531115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19395e7-24f5-4fa1-bf6f-20fb938feca7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:35.531951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19395e7-24f5-4fa1-bf6f-20fb938feca7 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:35.532249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19395e7-24f5-4fa1-bf6f-20fb938feca7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:35.532516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19395e7-24f5-4fa1-bf6f-20fb938feca7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:35.542322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19395e7-24f5-4fa1-bf6f-20fb938feca7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:35.543769 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19395e7-24f5-4fa1-bf6f-20fb938feca7 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:35.554207 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1324/2621] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:38.744736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:38.807830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91ebc0-b446-4fe2-aec3-20263f37cb7d None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:38.812330 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:38.812953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:38.813235 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:38.813509 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] PATH_INFO: `/v3/users/0b64c8cb38e64a38a70402c92a9bd6ed` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:38.814632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b64c8cb38e64a38a70402c92a9bd6ed)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:38.816576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:39.055714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b64c8cb38e64a38a70402c92a9bd6ed'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:39.056504 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b64c8cb38e64a38a70402c92a9bd6ed'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:39.057150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b64c8cb38e64a38a70402c92a9bd6ed'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:39.058338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:39.064541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b64c8cb38e64a38a70402c92a9bd6ed'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:39.161765 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b64c8cb38e64a38a70402c92a9bd6ed'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:39.179903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1298/2622] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:38 2022] DELETE /identity/v3/users/0b64c8cb38e64a38a70402c92a9bd6ed => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:39.184457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:39.184937 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:39.185089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:39.204118 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:39.205351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:39.266675 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1325/2623] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:39.276422 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:39.277820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:39.349552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f4e905-670d-4b2d-9a1a-275c59a49f43 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:39.350583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:39.351273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:39.351555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:39.351825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:39.362796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:39.364255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:39.374901 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1299/2624] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:39.860268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:39.861720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:39.876443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5b59d-2eee-4f9f-8286-40a9cf088688 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:39.882264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:39.883107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:39.883395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:39.883664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] PATH_INFO: `/v3/projects/a86b38f288f24ef1ac020b2327301845` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:39.884891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a86b38f288f24ef1ac020b2327301845)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:39.886785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:40.043391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a86b38f288f24ef1ac020b2327301845', 'request_id': 'req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.049093 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a86b38f288f24ef1ac020b2327301845', 'request_id': 'req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.056253 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a86b38f288f24ef1ac020b2327301845', 'request_id': 'req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.127001 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] The token cache is being invalidate because project a86b38f288f24ef1ac020b2327301845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:40.127270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.127924 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.129595 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1326/2625] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:39 2022] DELETE /identity/v3/projects/a86b38f288f24ef1ac020b2327301845 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:40.134679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:40.145502 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d45f15-cb34-45ed-90d1-965844a4e83b service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:40.202206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:40.202831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:40.203106 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:40.203376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] PATH_INFO: `/v3/users/186727cf1fbd4869996fe75c98f60cc1` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:40.204558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=186727cf1fbd4869996fe75c98f60cc1)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:40.206522 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:40.403832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186727cf1fbd4869996fe75c98f60cc1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.404455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186727cf1fbd4869996fe75c98f60cc1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.405062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186727cf1fbd4869996fe75c98f60cc1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.411887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186727cf1fbd4869996fe75c98f60cc1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.515652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-97b29b70-32dd-4eab-8391-932d1f7b93ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186727cf1fbd4869996fe75c98f60cc1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:40.521309 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:40.533566 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1300/2626] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:40 2022] DELETE /identity/v3/users/186727cf1fbd4869996fe75c98f60cc1 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:40.985724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:40.987194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:41.000786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc31d1-ecaf-4d03-aff7-6534ff1b2820 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.013195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:41.058814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.059455 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.059733 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.060223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] PATH_INFO: `/v3/projects/b6ba13b5f2464c6cb47821db7499a7e4` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.061353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ba13b5f2464c6cb47821db7499a7e4)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.063281 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.173441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ba13b5f2464c6cb47821db7499a7e4', 'request_id': 'req-5063cec1-e01f-4763-b050-6b227dde0356'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:41.179720 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ba13b5f2464c6cb47821db7499a7e4', 'request_id': 'req-5063cec1-e01f-4763-b050-6b227dde0356'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:41.185786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ba13b5f2464c6cb47821db7499a7e4', 'request_id': 'req-5063cec1-e01f-4763-b050-6b227dde0356'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:41.257778 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] The token cache is being invalidate because project b6ba13b5f2464c6cb47821db7499a7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:41.257986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:41.258686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-5063cec1-e01f-4763-b050-6b227dde0356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:41.260270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1327/2627] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:40 2022] DELETE /identity/v3/projects/b6ba13b5f2464c6cb47821db7499a7e4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:41.265481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.269543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.269543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.285501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:41.286727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:41.347908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1301/2628] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:41.356442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.356730 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.357007 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.377217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:41.378427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:41.386504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1328/2629] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:41.390843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:41.400899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a9403-dc69-4040-9e8b-c90528b69dc7 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.406125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.406710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.407140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.407423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.408099 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:35:41.408401 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.410577 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.422730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1302/2630] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:35:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:35:41.427035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:41.437370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e87ee-e775-4d96-adc1-6266ef0e46a4 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.442056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.442700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.442981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.443251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.443883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.445820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.591835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1329/2631] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:41 2022] POST /identity/v3/projects => generated 401 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:41.596014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:41.606994 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94125fb5-8596-4e25-aaf0-3d34321b8b2f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.664463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.665071 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.665349 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.665616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.666245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.668158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.777223 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1303/2632] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:41 2022] POST /identity/v3/users => generated 301 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:41.782809 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:41.789445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb3b0a-f008-465d-969e-a83c7462d93d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.794082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.794672 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.794948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.795214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.795869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.797488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.807508 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1330/2633] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:41.811408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:41.821799 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070a55c-b69f-4f23-93ff-4a7df2c1a4c4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.826423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d97f41-0c9f-4e39-8457-28a6c80108a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.827011 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d97f41-0c9f-4e39-8457-28a6c80108a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.827283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d97f41-0c9f-4e39-8457-28a6c80108a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.827550 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d97f41-0c9f-4e39-8457-28a6c80108a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.828211 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d97f41-0c9f-4e39-8457-28a6c80108a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.829902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d97f41-0c9f-4e39-8457-28a6c80108a0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.841166 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1304/2634] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:41.845243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:41.856144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdba0839-d251-4324-9908-5c698a59b42b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:41.860700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:41.861336 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.861608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.861873 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] PATH_INFO: `/v3/projects/ecf94f9af8e64c61bb2299f7017ee5ac/users/9ccd83b3c1154c9682b4f44aa8797e9b/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:41.876597 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecf94f9af8e64c61bb2299f7017ee5ac, user_id=9ccd83b3c1154c9682b4f44aa8797e9b, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:41.882128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:41.984339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1331/2635] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:41 2022] PUT /identity/v3/projects/ecf94f9af8e64c61bb2299f7017ee5ac/users/9ccd83b3c1154c9682b4f44aa8797e9b/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:41.994676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:41.994961 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:41.995233 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.023032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] MFA Rules not processed for user `9ccd83b3c1154c9682b4f44aa8797e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:42.024274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:42.063688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:42.098338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1305/2636] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:41 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:42.101156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.145422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:42.162460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa03733-bd3b-4dfd-aad6-6f645357094d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.167311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.171236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.171236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.171236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.171236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.171236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.253854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1332/2637] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:42 2022] POST /identity/v3/projects => generated 401 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:42.259356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.270311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c1b65-445c-42b7-95a9-e702a3f8d96e None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.327526 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.328164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.328444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.328712 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.329403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.331305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.413374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1306/2638] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:42 2022] POST /identity/v3/users => generated 307 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:42.420369 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.431537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcf74a-8310-47d2-9eb6-7d9973c8a7be admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.436142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.436726 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.437002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.437265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.437885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.439451 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.450320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1333/2639] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:42.454404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.464770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda3582-7b71-4cf6-ac8f-ad392e9265a5 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.469402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.470067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.470345 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.470623 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] PATH_INFO: `/v3/projects/24a4bb06bfa947279f9091824798cc6e/users/13cc19795ecb433592595efdae4e827e/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.485132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a4bb06bfa947279f9091824798cc6e, user_id=13cc19795ecb433592595efdae4e827e, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.487742 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.578770 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1307/2640] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:42 2022] PUT /identity/v3/projects/24a4bb06bfa947279f9091824798cc6e/users/13cc19795ecb433592595efdae4e827e/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:42.582718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.593268 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b3ea07-1bc4-4415-a446-24aff34b4513 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.650060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b559cc-7f99-40ee-8414-f94238c534f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.650590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b559cc-7f99-40ee-8414-f94238c534f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.650876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b559cc-7f99-40ee-8414-f94238c534f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.651152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b559cc-7f99-40ee-8414-f94238c534f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.651784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b559cc-7f99-40ee-8414-f94238c534f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.653408 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b559cc-7f99-40ee-8414-f94238c534f5 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.663383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1334/2641] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:42.667588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.677705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade2af4-aa29-4739-ad6e-2ab9f430a628 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.683048 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.683692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.683972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.684279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] PATH_INFO: `/v3/projects/24a4bb06bfa947279f9091824798cc6e/users/13cc19795ecb433592595efdae4e827e/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.686175 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=24a4bb06bfa947279f9091824798cc6e, user_id=13cc19795ecb433592595efdae4e827e, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.688826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.698960 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1308/2642] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:42 2022] PUT /identity/v3/projects/24a4bb06bfa947279f9091824798cc6e/users/13cc19795ecb433592595efdae4e827e/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:42.766278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.766557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.766827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.795689 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] MFA Rules not processed for user `13cc19795ecb433592595efdae4e827e`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:42.796746 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:42.868047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1335/2643] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:42 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:42.878633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:42.880380 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:42.937707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a5050-6f60-47b3-bbd3-e72ce2fec7eb admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:42.941478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:42.942082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:42.942359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:42.942632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:42.951703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:42.953162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:42.962899 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1309/2644] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:42 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:43.198764 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:43.208418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8549b-e49c-40b3-8f02-6f5e69cfbf0d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:43.213053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:43.213660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:43.213939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:43.214209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:43.247792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:43.270214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:43.271641 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:43.281168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1336/2645] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:43 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:47.535066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:47.536519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:47.579647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:47.595489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fca80e9-bfe4-4001-9663-aeb9fc776b27 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:47.599075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:47.599669 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:47.599948 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:47.600312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:47.602051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:47.669535 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:47.670988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bfc12-6f7f-4ba2-b571-43f5a3953ede service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:47.681324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1310/2646] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:49.419835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:49.433796 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e004694-3540-402f-bb40-eb415d9a6abb service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:49.439371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:49.440860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:49.441307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:49.441697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] PATH_INFO: `/v3/users/9ccd83b3c1154c9682b4f44aa8797e9b` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:49.444095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ccd83b3c1154c9682b4f44aa8797e9b)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:49.448972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:49.649137 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ccd83b3c1154c9682b4f44aa8797e9b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:49.649750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ccd83b3c1154c9682b4f44aa8797e9b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:49.650439 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ccd83b3c1154c9682b4f44aa8797e9b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:49.658577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ccd83b3c1154c9682b4f44aa8797e9b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:49.704984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ccd83b3c1154c9682b4f44aa8797e9b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:49.728942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1337/2647] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:49 2022] DELETE /identity/v3/users/9ccd83b3c1154c9682b4f44aa8797e9b => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:49.734597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:49.735140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:49.735140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:49.756479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:49.757887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:49.821172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1311/2648] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:49.832752 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:49.834233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:49.890206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:49.892372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:49.907293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b139dee-54ca-402b-b1cb-1db835d241c8 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:49.912697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:49.913317 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd7c5b-2587-4fc8-b16d-216c3ac6ec03 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:49.913958 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:49.914407 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:49.914881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] PATH_INFO: `/v3/users/7b0e07f56c484aca8c8b9cb900b7a3ec` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:49.916323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0e07f56c484aca8c8b9cb900b7a3ec)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:49.918430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2c934-fb29-4bf6-9267-7a4c21a23aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:49.918812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:49.919560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2c934-fb29-4bf6-9267-7a4c21a23aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:49.919841 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2c934-fb29-4bf6-9267-7a4c21a23aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:49.920168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2c934-fb29-4bf6-9267-7a4c21a23aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:49.943198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2c934-fb29-4bf6-9267-7a4c21a23aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:49.944737 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2c934-fb29-4bf6-9267-7a4c21a23aa6 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:49.956163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1338/2649] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:50.150893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0e07f56c484aca8c8b9cb900b7a3ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.151675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0e07f56c484aca8c8b9cb900b7a3ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.152776 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0e07f56c484aca8c8b9cb900b7a3ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.161439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0e07f56c484aca8c8b9cb900b7a3ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.247640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0e07f56c484aca8c8b9cb900b7a3ec'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.266771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:50.273019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1312/2650] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:49 2022] DELETE /identity/v3/users/7b0e07f56c484aca8c8b9cb900b7a3ec => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:50.277663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:50.278236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:50.279314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:50.303789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:50.305332 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:50.361121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:50.376681 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1339/2651] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:50.394861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:50.394861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:50.442860 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:50.449132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:50.467277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02779f85-dc70-4f5d-a836-6a4d3363fbdd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:50.469364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b132a-fe16-41aa-b9e3-dd0a426ac1f0 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:50.478507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:50.478744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:50.478907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:50.479015 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:50.479116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:50.479215 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:50.479391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] PATH_INFO: `/v3/projects/ecf94f9af8e64c61bb2299f7017ee5ac` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:50.479498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:50.479596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecf94f9af8e64c61bb2299f7017ee5ac)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:50.479925 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:50.491100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:50.492773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:50.507582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1313/2652] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:50.664518 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecf94f9af8e64c61bb2299f7017ee5ac', 'request_id': 'req-336e32ad-1c12-4352-891c-e1a9ad6b59e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.669977 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecf94f9af8e64c61bb2299f7017ee5ac', 'request_id': 'req-336e32ad-1c12-4352-891c-e1a9ad6b59e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.676201 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecf94f9af8e64c61bb2299f7017ee5ac', 'request_id': 'req-336e32ad-1c12-4352-891c-e1a9ad6b59e8'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.802245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] The token cache is being invalidate because project ecf94f9af8e64c61bb2299f7017ee5ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:50.802590 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.803309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:50.809654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1340/2653] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:50 2022] DELETE /identity/v3/projects/ecf94f9af8e64c61bb2299f7017ee5ac => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:50.813374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:50.823829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a2cd5-1403-41b2-9d5f-1d685de8eb15 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:50.914769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:50.915541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:50.915922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:50.916321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] PATH_INFO: `/v3/users/13cc19795ecb433592595efdae4e827e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:50.920939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13cc19795ecb433592595efdae4e827e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:50.927477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:51.454043 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:51.456582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:51.460285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13cc19795ecb433592595efdae4e827e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.460983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13cc19795ecb433592595efdae4e827e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.461650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13cc19795ecb433592595efdae4e827e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.470826 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e32ad-1c12-4352-891c-e1a9ad6b59e8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:51.471055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13cc19795ecb433592595efdae4e827e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.477243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:51.477940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:51.478220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:51.478490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] PATH_INFO: `/v3/projects/9c1cb085c9914377898584966c934d33` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:51.479638 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1cb085c9914377898584966c934d33)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:51.481612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:51.801786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13cc19795ecb433592595efdae4e827e'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.822409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1314/2654] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:50 2022] DELETE /identity/v3/users/13cc19795ecb433592595efdae4e827e => generated 0 bytes in 1010 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:51.852965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1cb085c9914377898584966c934d33', 'request_id': 'req-02e9a4a9-c788-4bf1-8161-d83c96557d04'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.859460 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1cb085c9914377898584966c934d33', 'request_id': 'req-02e9a4a9-c788-4bf1-8161-d83c96557d04'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.866571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1cb085c9914377898584966c934d33', 'request_id': 'req-02e9a4a9-c788-4bf1-8161-d83c96557d04'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.986009 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] The token cache is being invalidate because project 9c1cb085c9914377898584966c934d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:51.986419 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.987552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:51.989644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1341/2655] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:51 2022] DELETE /identity/v3/projects/9c1cb085c9914377898584966c934d33 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:51.993942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:51.996601 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:52.028204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:52.086701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b9aeb3-b187-4197-96c3-d52805f4b640 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:52.153346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:52.153987 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:52.154326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:52.154537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] PATH_INFO: `/v3/users/76560436e33842aaba8206737b917737` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:52.155672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76560436e33842aaba8206737b917737)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:52.157627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:52.373987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:52.376340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:52.403774 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9a4a9-c788-4bf1-8161-d83c96557d04 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:52.410301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76560436e33842aaba8206737b917737'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.411331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76560436e33842aaba8206737b917737'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.412312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76560436e33842aaba8206737b917737'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.418147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:52.418791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:52.419062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:52.419324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] PATH_INFO: `/v3/users/0782e27e2318438fbf155dfa9cb834b6` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:52.424147 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0782e27e2318438fbf155dfa9cb834b6)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:52.424918 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76560436e33842aaba8206737b917737'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.426062 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:52.477412 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76560436e33842aaba8206737b917737'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.501205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1315/2656] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:51 2022] DELETE /identity/v3/users/76560436e33842aaba8206737b917737 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:52.502310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:52.503683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:52.617209 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc123cf0-27d5-42e4-b1fd-e9b493c5abf6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:52.623620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:52.624200 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:52.624486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:52.624758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] PATH_INFO: `/v3/projects/24a4bb06bfa947279f9091824798cc6e` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:52.626131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24a4bb06bfa947279f9091824798cc6e)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:52.628451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:52.667707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0782e27e2318438fbf155dfa9cb834b6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.669334 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0782e27e2318438fbf155dfa9cb834b6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.670167 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0782e27e2318438fbf155dfa9cb834b6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.675218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:52.678142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0782e27e2318438fbf155dfa9cb834b6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.752597 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24a4bb06bfa947279f9091824798cc6e', 'request_id': 'req-8369d71f-35d7-486c-abc4-05a305801121'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.758819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24a4bb06bfa947279f9091824798cc6e', 'request_id': 'req-8369d71f-35d7-486c-abc4-05a305801121'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.762541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0782e27e2318438fbf155dfa9cb834b6'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.766238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24a4bb06bfa947279f9091824798cc6e', 'request_id': 'req-8369d71f-35d7-486c-abc4-05a305801121'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.784686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1342/2657] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:52 2022] DELETE /identity/v3/users/0782e27e2318438fbf155dfa9cb834b6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:52.891660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] The token cache is being invalidate because project 24a4bb06bfa947279f9091824798cc6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:52.892341 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.892670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:52.894485 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1316/2658] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:52 2022] DELETE /identity/v3/projects/24a4bb06bfa947279f9091824798cc6e => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:53.038694 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.040636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:53.079356 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.082335 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:53.135368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3fdfb-572f-452d-a821-8c601bbb53b0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.140707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.141505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.141854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.142194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] PATH_INFO: `/v3/projects/c29fc03a820d453eab6ec70979530bfc` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.143608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c29fc03a820d453eab6ec70979530bfc)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.146106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.193073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8369d71f-35d7-486c-abc4-05a305801121 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.197736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.198616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.198739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.199004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] PATH_INFO: `/v3/projects/0b2a3c6102fb4fe5bd9c9ef1b8f8e78b` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.200202 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2a3c6102fb4fe5bd9c9ef1b8f8e78b)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.202083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.209955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:53.242708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c29fc03a820d453eab6ec70979530bfc', 'request_id': 'req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.248827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c29fc03a820d453eab6ec70979530bfc', 'request_id': 'req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.254869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c29fc03a820d453eab6ec70979530bfc', 'request_id': 'req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.304521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2a3c6102fb4fe5bd9c9ef1b8f8e78b', 'request_id': 'req-aa8be486-cd43-4af5-a91f-53bb6959d846'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.312670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2a3c6102fb4fe5bd9c9ef1b8f8e78b', 'request_id': 'req-aa8be486-cd43-4af5-a91f-53bb6959d846'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.315012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] The token cache is being invalidate because project c29fc03a820d453eab6ec70979530bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:53.321824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.321824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-828bc25e-9b81-4861-b1bd-9cfe2fec9ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.321824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1343/2659] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:53 2022] DELETE /identity/v3/projects/c29fc03a820d453eab6ec70979530bfc => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:53.322274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2a3c6102fb4fe5bd9c9ef1b8f8e78b', 'request_id': 'req-aa8be486-cd43-4af5-a91f-53bb6959d846'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.322389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940eab0-25f2-4a1b-a26f-0f73897077ca None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.322521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940eab0-25f2-4a1b-a26f-0f73897077ca None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.322854 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8940eab0-25f2-4a1b-a26f-0f73897077ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.342609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8940eab0-25f2-4a1b-a26f-0f73897077ca None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:53.348551 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8940eab0-25f2-4a1b-a26f-0f73897077ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:53.400647 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] The token cache is being invalidate because project 0b2a3c6102fb4fe5bd9c9ef1b8f8e78b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:53.400941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.401684 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1344/2660] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:53.406570 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-aa8be486-cd43-4af5-a91f-53bb6959d846 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:53.411543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1317/2661] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:53 2022] DELETE /identity/v3/projects/0b2a3c6102fb4fe5bd9c9ef1b8f8e78b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:53.411704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.411704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.411704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.414874 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11aefa-9e3e-4c9d-9017-d4b819e6ed18 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.415152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11aefa-9e3e-4c9d-9017-d4b819e6ed18 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.415451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11aefa-9e3e-4c9d-9017-d4b819e6ed18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.429126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:53.430356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:53.436569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2e11aefa-9e3e-4c9d-9017-d4b819e6ed18 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:53.437730 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2e11aefa-9e3e-4c9d-9017-d4b819e6ed18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:53.439036 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1345/2662] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:53.445709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1318/2663] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:53.449397 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.454140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.454525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.454873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.458722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9323272-e3ad-4cda-b932-42aaec348328 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.464594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.465293 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.465668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.466035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.466797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:35:53.467220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.468976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.475875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:53.490033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:53.490182 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1346/2664] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:35:53.490295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1319/2665] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:53.492793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.492938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.502981 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde78ba7-1b2b-4fad-ab36-dc0257a8f273 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.503168 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1b286-40b7-40be-9dfa-ad612cbc902b admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.508075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.508239 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.508801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.509084 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.509355 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.509643 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.509974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.510483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.511053 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.511809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:35:53.511985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.512318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.514018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.526220 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1320/2666] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:35:53.530517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.540819 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a69eb94-8e67-45be-9797-3b4686c6bc84 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.546135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.546714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.546992 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.547260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.547868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.549884 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.602728 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1347/2667] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:53.607650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.647219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1321/2668] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:53.648508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.668948 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2fc8c7-03ce-4589-ab12-4f774984c739 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.713893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c1e20-b2f4-4da1-aab4-40520f39d5ee admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.718702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.719392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.719677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.719952 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.720605 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.722518 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.726228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.727148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.727433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.727709 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.728370 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.730815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.741238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:53.807867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1322/2669] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:53.812278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.818939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1348/2670] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:53 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:53.823795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fa6aa-c902-4e68-bfdd-bfb69a2dadbd admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.825998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.828710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.829392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.829791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.830069 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.830706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.832372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.834158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52b092-7a17-471a-9933-73c9d1e7d073 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.839482 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.840105 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.840390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.840725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.841366 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.843010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.843651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1323/2671] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:35:53.847624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.859243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1349/2672] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:35:53.859421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0a868-4d94-44fe-8dd8-6576ba943ea3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.860206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.863919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.864580 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.864869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.865153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.865832 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.867423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.870755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3e52e-db71-452a-8104-007515e55778 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.875584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.876271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.876583 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.876878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.877605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.878998 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1324/2673] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:35:53.879241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.883488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.891688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1350/2674] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:35:53.894375 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2661744-0a79-48c6-820d-8376e44d7fc1 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.895680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:53.899055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53458a1-1a5c-403a-bda8-cc608f66fc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.899698 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53458a1-1a5c-403a-bda8-cc608f66fc36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.899978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53458a1-1a5c-403a-bda8-cc608f66fc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.900293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53458a1-1a5c-403a-bda8-cc608f66fc36 admin admin] PATH_INFO: `/v3/projects/207534d06b7742dc8502520a575f35d7/users/29a6d31b89a64899aac054c2f74ed94a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.906829 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fd81a-55c3-4191-a9cf-2295c08634d8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:53.911899 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f063dac-99d9-48e9-bf96-36514d9a0a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:53.916022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063dac-99d9-48e9-bf96-36514d9a0a80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:53.916022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063dac-99d9-48e9-bf96-36514d9a0a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:53.916022 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f063dac-99d9-48e9-bf96-36514d9a0a80 admin admin] PATH_INFO: `/v3/projects/f2a7bcc9b9b449fbb78e77e743d49fc7/users/ac468dda03b54355a77d5feb0b7aa176/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:53.916338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53458a1-1a5c-403a-bda8-cc608f66fc36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207534d06b7742dc8502520a575f35d7, user_id=29a6d31b89a64899aac054c2f74ed94a, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.919444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53458a1-1a5c-403a-bda8-cc608f66fc36 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:53.929246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f063dac-99d9-48e9-bf96-36514d9a0a80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a7bcc9b9b449fbb78e77e743d49fc7, user_id=ac468dda03b54355a77d5feb0b7aa176, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:53.931965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f063dac-99d9-48e9-bf96-36514d9a0a80 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.013164 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1325/2675] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:53 2022] PUT /identity/v3/projects/207534d06b7742dc8502520a575f35d7/users/29a6d31b89a64899aac054c2f74ed94a/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:54.018839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:54.024493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.032131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1351/2676] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:53 2022] PUT /identity/v3/projects/f2a7bcc9b9b449fbb78e77e743d49fc7/users/ac468dda03b54355a77d5feb0b7aa176/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:54.032275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.032275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.041358 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.042238 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.042516 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.053588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] MFA Rules not processed for user `29a6d31b89a64899aac054c2f74ed94a`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:54.054921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:54.069078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] MFA Rules not processed for user `ac468dda03b54355a77d5feb0b7aa176`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:54.070254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:54.120045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1326/2677] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:35:54.132275 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.133624 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:54.134569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1352/2678] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:54.139519 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.192773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fec91b-d853-417d-ae48-55ad163cc2e3 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.196558 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.197236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.197515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.198284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.200793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bfde3b-24e2-4fa2-8171-b17beb93a304 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.205399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.205999 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.206290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.206555 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.206908 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.207186 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.208506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.209089 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.218850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1327/2679] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:35:54.241530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:54.300569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1353/2680] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:35:54 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:35:54.304789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.315856 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3df11-817c-485e-8d13-71106fa3e9c7 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.372680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.373168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.373442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.373710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.374328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.376333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.453128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:54.499304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1328/2681] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:35:54 2022] POST /identity/v3/users => generated 296 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:35:54.503099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.513776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687cf8-9792-4878-a185-93952814a18f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.519414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.520025 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.520308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.520580 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.521217 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.522810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.534019 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1354/2682] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:54.538206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.548366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc0266-a193-4b6d-b391-5dd58eecb758 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.553277 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.553938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.554218 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.554488 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] PATH_INFO: `/v3/projects/a3aed682a50544e3b4241f398b7660e7/users/cf186d010ccb47b18edc97fa7ff0e457/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.569552 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3aed682a50544e3b4241f398b7660e7, user_id=cf186d010ccb47b18edc97fa7ff0e457, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.572331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.669541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1329/2683] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:54 2022] PUT /identity/v3/projects/a3aed682a50544e3b4241f398b7660e7/users/cf186d010ccb47b18edc97fa7ff0e457/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:35:54.673490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.683900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dde331-831d-49c7-81db-36874136aeea admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.744059 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6635a907-ef17-451f-9396-67730e24a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.745267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635a907-ef17-451f-9396-67730e24a7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.745546 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635a907-ef17-451f-9396-67730e24a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.745816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6635a907-ef17-451f-9396-67730e24a7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.746457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635a907-ef17-451f-9396-67730e24a7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.748068 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6635a907-ef17-451f-9396-67730e24a7d6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.758444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1355/2684] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:35:54.763479 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.773644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee9735-62a7-42b5-83e9-e398c722d0f2 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.778935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.779576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.779854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.780161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] PATH_INFO: `/v3/projects/a3aed682a50544e3b4241f398b7660e7/users/cf186d010ccb47b18edc97fa7ff0e457/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.782312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3aed682a50544e3b4241f398b7660e7, user_id=cf186d010ccb47b18edc97fa7ff0e457, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.785043 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:54.796023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1330/2685] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:35:54 2022] PUT /identity/v3/projects/a3aed682a50544e3b4241f398b7660e7/users/cf186d010ccb47b18edc97fa7ff0e457/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:54.806845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.807312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.807457 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.826835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:54.835234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] MFA Rules not processed for user `cf186d010ccb47b18edc97fa7ff0e457`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:54.836486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:54.906175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1356/2686] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:54.920105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:54.974152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:54.977400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc94e65-d5b4-4aab-b46b-3abecda6129f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:54.981771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:54.982392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:54.982678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:54.982949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:54.992362 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:54.993846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:55.003978 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1331/2687] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:55.114490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:55.124066 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6fc33d-2c19-4845-93a3-590b932cd6d5 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:55.127715 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f35cb4-20ab-43b4-b07d-7602b7c7eefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:55.128284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f35cb4-20ab-43b4-b07d-7602b7c7eefa service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:55.128570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f35cb4-20ab-43b4-b07d-7602b7c7eefa service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:55.128847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f35cb4-20ab-43b4-b07d-7602b7c7eefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:55.185969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f35cb4-20ab-43b4-b07d-7602b7c7eefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:55.187386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f35cb4-20ab-43b4-b07d-7602b7c7eefa service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:55.196090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1357/2688] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:58.376861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:58.456857 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f3cb3a-a27d-4742-8df6-03ab76f66b2c service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:58.461393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:58.462016 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:58.462289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:58.462600 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] PATH_INFO: `/v3/users/ac468dda03b54355a77d5feb0b7aa176` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:58.465038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac468dda03b54355a77d5feb0b7aa176)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:58.467472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:58.660920 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac468dda03b54355a77d5feb0b7aa176'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:58.661576 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac468dda03b54355a77d5feb0b7aa176'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:58.662172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac468dda03b54355a77d5feb0b7aa176'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:58.669989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac468dda03b54355a77d5feb0b7aa176'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:58.743501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac468dda03b54355a77d5feb0b7aa176'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:58.761239 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1332/2689] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:58 2022] DELETE /identity/v3/users/ac468dda03b54355a77d5feb0b7aa176 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:58.765749 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:58.766252 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:58.766373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:58.784724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:35:58.785913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:58.843872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1358/2690] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:35:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:35:58.853760 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:58.855702 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:58.914492 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a918c682-2d9b-473c-8480-bb4c60c0fc64 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:58.918415 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:58.919040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:58.919316 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:58.919617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:58.928517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:58.932849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:58.939905 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1333/2691] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:35:59.334270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:59.335759 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:35:59.350624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ce9ea-5fe2-4697-8e6b-655a9a9a9859 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:59.356320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:59.357581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:59.357864 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:59.358133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] PATH_INFO: `/v3/projects/f2a7bcc9b9b449fbb78e77e743d49fc7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:59.359243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a7bcc9b9b449fbb78e77e743d49fc7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:59.365172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:59.495699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:59.513040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a7bcc9b9b449fbb78e77e743d49fc7', 'request_id': 'req-49a84f89-1d75-441c-8cd5-b87105c0cca3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.518834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a7bcc9b9b449fbb78e77e743d49fc7', 'request_id': 'req-49a84f89-1d75-441c-8cd5-b87105c0cca3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.525503 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a7bcc9b9b449fbb78e77e743d49fc7', 'request_id': 'req-49a84f89-1d75-441c-8cd5-b87105c0cca3'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.612634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] The token cache is being invalidate because project f2a7bcc9b9b449fbb78e77e743d49fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:35:59.613039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.613914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.615478 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1359/2692] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:35:59 2022] DELETE /identity/v3/projects/f2a7bcc9b9b449fbb78e77e743d49fc7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:35:59.619468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:35:59.629097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6136506c-d6cc-49a2-ac82-038dbfce48fc service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:35:59.638540 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:35:59.695480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:35:59.696296 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:35:59.696616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:35:59.696979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] PATH_INFO: `/v3/users/cf186d010ccb47b18edc97fa7ff0e457` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:35:59.698162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf186d010ccb47b18edc97fa7ff0e457)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:35:59.700690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:35:59.861500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf186d010ccb47b18edc97fa7ff0e457'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.862152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf186d010ccb47b18edc97fa7ff0e457'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.862796 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf186d010ccb47b18edc97fa7ff0e457'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:35:59.871878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf186d010ccb47b18edc97fa7ff0e457'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.001729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-885f3daa-2c1b-45bb-9d6b-08c5adeeeccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf186d010ccb47b18edc97fa7ff0e457'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.020559 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1334/2693] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:35:59 2022] DELETE /identity/v3/users/cf186d010ccb47b18edc97fa7ff0e457 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:00.523114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:00.524612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:00.540500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a84f89-1d75-441c-8cd5-b87105c0cca3 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:00.602998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:00.602998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:00.602998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:00.602998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] PATH_INFO: `/v3/projects/a3aed682a50544e3b4241f398b7660e7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:00.603412 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3aed682a50544e3b4241f398b7660e7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:00.604704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:00.726970 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3aed682a50544e3b4241f398b7660e7', 'request_id': 'req-486d5943-08b9-4d65-84af-0fe041e56c55'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.732634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3aed682a50544e3b4241f398b7660e7', 'request_id': 'req-486d5943-08b9-4d65-84af-0fe041e56c55'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.738692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3aed682a50544e3b4241f398b7660e7', 'request_id': 'req-486d5943-08b9-4d65-84af-0fe041e56c55'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.806221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] The token cache is being invalidate because project a3aed682a50544e3b4241f398b7660e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:00.806446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.807071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-486d5943-08b9-4d65-84af-0fe041e56c55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:00.809133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1360/2694] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:00 2022] DELETE /identity/v3/projects/a3aed682a50544e3b4241f398b7660e7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:00.813959 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:00.814420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:00.814519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:00.834894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:00.836149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:00.895990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1335/2695] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:00.904307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:00.904654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:00.904927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:00.925294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:00.926489 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:00.935421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1361/2696] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:00.939638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:00.949514 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e20d19-f5ed-489c-9ae9-ffc5b2fa3028 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:00.954008 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:00.954675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:00.954875 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:00.955161 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:00.955783 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:00.956120 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:00.957738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:00.969701 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1336/2697] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:00.974106 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:00.984969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e081e6-c997-474f-9383-3a6b993c2a5e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:00.989576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:00.990119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:00.990392 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:00.990657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:00.991279 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:00.993331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.099907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1362/2698] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:00 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:01.104081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:01.114125 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9154f084-4e3e-4b6f-9da1-7d732e96ddee admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:01.131292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:01.170336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:01.170937 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.171210 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.171486 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.172149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:01.174050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.253249 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1337/2699] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:01 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:01.257381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:01.268254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e5895-e213-4acd-a300-4d7d7b870c9d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:01.272944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:01.273530 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.273805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.274072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.274704 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:01.276349 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.286894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1363/2700] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:01.293111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:01.303067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15841a3a-4907-496d-9bb1-70a468448380 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:01.308972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388cdc2-e3df-4ab8-ac9b-faeb9336095c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:01.309561 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388cdc2-e3df-4ab8-ac9b-faeb9336095c admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.309837 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388cdc2-e3df-4ab8-ac9b-faeb9336095c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.310111 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388cdc2-e3df-4ab8-ac9b-faeb9336095c admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.310755 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388cdc2-e3df-4ab8-ac9b-faeb9336095c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:01.312439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388cdc2-e3df-4ab8-ac9b-faeb9336095c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.322882 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1338/2701] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:01.327798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:01.337731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08319173-7cdc-4688-bb1b-31baea93cfeb admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:01.342748 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:01.343388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.343668 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.343942 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] PATH_INFO: `/v3/projects/ea8651ebdb9946a5bee59f0ce0906698/users/9880de95d0ec43d29cd48650aab3ccff/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.346142 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:01.358687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8651ebdb9946a5bee59f0ce0906698, user_id=9880de95d0ec43d29cd48650aab3ccff, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:01.361387 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.412859 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1364/2702] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:01 2022] PUT /identity/v3/projects/ea8651ebdb9946a5bee59f0ce0906698/users/9880de95d0ec43d29cd48650aab3ccff/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:01.423494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.423781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.424100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.452633 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] MFA Rules not processed for user `9880de95d0ec43d29cd48650aab3ccff`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:01.453836 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:01.515729 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1339/2703] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:01.526851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:01.528791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:01.585318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5557be-98e3-42a0-8e3e-88ee7a809488 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:01.589197 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3bcdcd-9c3f-4353-9090-3a51dd5be0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:01.589902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bcdcd-9c3f-4353-9090-3a51dd5be0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.590171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bcdcd-9c3f-4353-9090-3a51dd5be0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.590438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3bcdcd-9c3f-4353-9090-3a51dd5be0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.598579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bcdcd-9c3f-4353-9090-3a51dd5be0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:01.600015 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3bcdcd-9c3f-4353-9090-3a51dd5be0d6 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.608473 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1365/2704] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:01.627419 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:01.695285 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ccbbf-d4ea-4bbe-a8b2-424167e145af None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:01.700583 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:01.701204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:01.703471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:01.703471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] PATH_INFO: `/v3/users/9880de95d0ec43d29cd48650aab3ccff` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:01.703471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9880de95d0ec43d29cd48650aab3ccff)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:01.704818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:01.928185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:01.940989 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9880de95d0ec43d29cd48650aab3ccff'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:01.941640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9880de95d0ec43d29cd48650aab3ccff'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:01.942242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9880de95d0ec43d29cd48650aab3ccff'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:01.949423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9880de95d0ec43d29cd48650aab3ccff'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.093155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9880de95d0ec43d29cd48650aab3ccff'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.111538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1340/2705] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:01 2022] DELETE /identity/v3/users/9880de95d0ec43d29cd48650aab3ccff => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:02.116192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:02.116532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:02.116781 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:02.135926 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:02.137686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:02.196840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1366/2706] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:02.207549 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:02.209039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:02.267307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c76b8-6034-4be3-b658-5a96670029e4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:02.270865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ca03c-ad30-44fc-a9ea-82b1687715f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:02.271465 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ca03c-ad30-44fc-a9ea-82b1687715f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:02.271741 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ca03c-ad30-44fc-a9ea-82b1687715f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:02.272065 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ca03c-ad30-44fc-a9ea-82b1687715f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:02.281619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ca03c-ad30-44fc-a9ea-82b1687715f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:02.283036 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ca03c-ad30-44fc-a9ea-82b1687715f7 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:02.292414 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1341/2707] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:02.735291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:02.736868 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:02.749389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9eee1-d799-4453-9393-c8a6799cfaaf None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:02.754046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:02.754680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:02.754957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:02.755228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] PATH_INFO: `/v3/projects/ea8651ebdb9946a5bee59f0ce0906698` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:02.756376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8651ebdb9946a5bee59f0ce0906698)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:02.758306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:02.858212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8651ebdb9946a5bee59f0ce0906698', 'request_id': 'req-a25611c1-763a-40b3-a044-3534c368d122'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.863724 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8651ebdb9946a5bee59f0ce0906698', 'request_id': 'req-a25611c1-763a-40b3-a044-3534c368d122'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.870075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8651ebdb9946a5bee59f0ce0906698', 'request_id': 'req-a25611c1-763a-40b3-a044-3534c368d122'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.943343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] The token cache is being invalidate because project ea8651ebdb9946a5bee59f0ce0906698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:02.943734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.944621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-a25611c1-763a-40b3-a044-3534c368d122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:02.946362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1367/2708] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:02 2022] DELETE /identity/v3/projects/ea8651ebdb9946a5bee59f0ce0906698 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:02.951714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:02.952204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:02.952302 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:02.972082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:02.973283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:03.032763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1342/2709] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:03.041347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.041632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.041907 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.061943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:03.063160 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:03.071017 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1368/2710] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:03.076679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.089260 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c95978-bdc6-44de-a9d5-e47f13267c56 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.102413 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.103093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.103409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.103688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.104451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:03.104749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.107281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.119820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1343/2711] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:03.124598 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.129049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:03.134164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ccae71-63bd-4592-870d-d13389bb6785 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.139459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.140090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.140376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.140767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.141402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.143285 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.252923 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1369/2712] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:03 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:03.257090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.267391 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c73f00-42d2-4cc2-8807-0cd368954ffc admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.300721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:03.323147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.323752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.324749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.332466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.332466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.332466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.454358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1344/2713] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:03 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:03.458418 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.469533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c4d17-2a5b-42ec-bbf0-3c1ec32c2b89 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.474191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.474791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.475100 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.475376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.476027 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.477629 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.487966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1370/2714] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:03.498734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.508582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee4497-c9f6-46f2-82e1-78c4eb5ff6a6 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.513225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a69d83-b7fb-4b2c-8ee8-2f9b5f7a597f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.513811 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69d83-b7fb-4b2c-8ee8-2f9b5f7a597f admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.514115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69d83-b7fb-4b2c-8ee8-2f9b5f7a597f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.514387 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a69d83-b7fb-4b2c-8ee8-2f9b5f7a597f admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.515080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a69d83-b7fb-4b2c-8ee8-2f9b5f7a597f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.516689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a69d83-b7fb-4b2c-8ee8-2f9b5f7a597f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.527545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1345/2715] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:03.532038 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.541957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b71d05-3180-401b-911f-3c0b73a18cb7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.547088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.547731 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.548031 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.548302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] PATH_INFO: `/v3/projects/bbdc6c4dec2541b4ae7347a3b33e9abf/users/d6327376bad4443b9f5a6da361e648a5/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.563792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbdc6c4dec2541b4ae7347a3b33e9abf, user_id=d6327376bad4443b9f5a6da361e648a5, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.566436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.654301 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1371/2716] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:03 2022] PUT /identity/v3/projects/bbdc6c4dec2541b4ae7347a3b33e9abf/users/d6327376bad4443b9f5a6da361e648a5/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:03.664803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.665090 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.665366 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.693168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] MFA Rules not processed for user `d6327376bad4443b9f5a6da361e648a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:03.694428 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:03.758602 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1346/2717] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:03.769779 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.771507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:03.827244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ef7dc-f799-4341-9a8f-91fcdd546a12 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.830892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb6cb-04bb-4eab-9917-1357f69f8728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.831485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb6cb-04bb-4eab-9917-1357f69f8728 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.831761 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb6cb-04bb-4eab-9917-1357f69f8728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.832049 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb6cb-04bb-4eab-9917-1357f69f8728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.841361 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcb6cb-04bb-4eab-9917-1357f69f8728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.842862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcb6cb-04bb-4eab-9917-1357f69f8728 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:03.851327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1372/2718] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:03.872845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:03.934562 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9a92a-9402-4b7f-9c09-4ef00a5a7320 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:03.939794 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:03.940450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:03.940734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:03.941088 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] PATH_INFO: `/v3/users/d6327376bad4443b9f5a6da361e648a5` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:03.942293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6327376bad4443b9f5a6da361e648a5)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:03.944515 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:04.077670 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6327376bad4443b9f5a6da361e648a5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.078295 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6327376bad4443b9f5a6da361e648a5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.078900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6327376bad4443b9f5a6da361e648a5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.080097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:04.086135 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6327376bad4443b9f5a6da361e648a5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.143955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6327376bad4443b9f5a6da361e648a5'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.163815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1347/2719] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:03 2022] DELETE /identity/v3/users/d6327376bad4443b9f5a6da361e648a5 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:04.168836 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:04.169120 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:04.169395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:04.190203 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:04.191449 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:04.257428 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1373/2720] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:04.268939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:04.270480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:04.337124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:04.338613 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90375ef-fe80-45b7-87b8-66bb98f4f10c admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:04.338832 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:04.342671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983006aa-22ff-465e-9f76-5f1b3218c787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:04.343281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983006aa-22ff-465e-9f76-5f1b3218c787 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:04.343572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983006aa-22ff-465e-9f76-5f1b3218c787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:04.343825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983006aa-22ff-465e-9f76-5f1b3218c787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:04.351144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e745df73-ace1-4236-b153-01052b69fe3c None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:04.352921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983006aa-22ff-465e-9f76-5f1b3218c787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:04.354582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983006aa-22ff-465e-9f76-5f1b3218c787 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:04.356236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:04.356862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:04.357146 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:04.357420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] PATH_INFO: `/v3/users/29a6d31b89a64899aac054c2f74ed94a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:04.358663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29a6d31b89a64899aac054c2f74ed94a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:04.360631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:04.364967 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1348/2721] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:04.533800 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:04.536312 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29a6d31b89a64899aac054c2f74ed94a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.536988 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29a6d31b89a64899aac054c2f74ed94a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.537611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29a6d31b89a64899aac054c2f74ed94a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.544852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29a6d31b89a64899aac054c2f74ed94a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.624165 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29a6d31b89a64899aac054c2f74ed94a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.643853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1374/2722] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:04 2022] DELETE /identity/v3/users/29a6d31b89a64899aac054c2f74ed94a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:04.648660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:04.648928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:04.649208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:04.669455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:04.670662 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:04.732449 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1349/2723] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:04.744861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:04.744861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:04.749906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:04.751584 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:04.763551 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8ac95-deb7-4afb-9dcb-fca4616d7458 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:04.768705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:04.769369 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:04.769654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:04.769949 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] PATH_INFO: `/v3/projects/bbdc6c4dec2541b4ae7347a3b33e9abf` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:04.771105 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbdc6c4dec2541b4ae7347a3b33e9abf)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:04.773033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:04.820039 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6eb68-7ecc-412a-b3a4-d6ec8d1a0680 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:04.823676 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e9133a-92c5-4d1c-b428-046d56820f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:04.824328 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9133a-92c5-4d1c-b428-046d56820f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:04.827280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9133a-92c5-4d1c-b428-046d56820f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:04.827280 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e9133a-92c5-4d1c-b428-046d56820f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:04.833842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9133a-92c5-4d1c-b428-046d56820f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:04.835205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e9133a-92c5-4d1c-b428-046d56820f91 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:04.845011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1375/2724] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:04.910445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbdc6c4dec2541b4ae7347a3b33e9abf', 'request_id': 'req-2cf94cba-5082-4e4e-b024-f6696006bb42'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.916709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbdc6c4dec2541b4ae7347a3b33e9abf', 'request_id': 'req-2cf94cba-5082-4e4e-b024-f6696006bb42'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.924040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbdc6c4dec2541b4ae7347a3b33e9abf', 'request_id': 'req-2cf94cba-5082-4e4e-b024-f6696006bb42'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:04.925247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:05.069619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] The token cache is being invalidate because project bbdc6c4dec2541b4ae7347a3b33e9abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:05.070007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.070733 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-2cf94cba-5082-4e4e-b024-f6696006bb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.072408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1350/2725] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:04 2022] DELETE /identity/v3/projects/bbdc6c4dec2541b4ae7347a3b33e9abf => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:05.077520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.077805 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.078077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.098569 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:05.099797 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.167421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1376/2726] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:05.175787 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.176327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.176431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.198093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:05.199914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.207977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1351/2727] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:05.212265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.222406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf87200-23e1-4e0e-8081-cfc5f871be9f None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.227648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.227791 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.228044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.230274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.230274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:05.230274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.231131 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.243132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1377/2728] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:05.248355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.257556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7d1e1f-e6b1-4916-88ab-25742d409b7c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.263374 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.263966 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.264971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.264971 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.265230 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.267081 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.322835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.325195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.339002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb70cba-f3f3-4eab-87a0-157dceb40ef0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.344158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.344798 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.345079 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.345353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] PATH_INFO: `/v3/projects/207534d06b7742dc8502520a575f35d7` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.346504 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=207534d06b7742dc8502520a575f35d7)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.348480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.352266 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1352/2729] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/projects => generated 359 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:05.356664 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.420265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353aa56a-a065-4c7f-b660-47c5f79ca60e admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.425042 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.425645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.425925 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.426195 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.426938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.428965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.445954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:05.449481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207534d06b7742dc8502520a575f35d7', 'request_id': 'req-b5c78b3d-2110-487b-a993-fbb6e02ae097'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.454845 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207534d06b7742dc8502520a575f35d7', 'request_id': 'req-b5c78b3d-2110-487b-a993-fbb6e02ae097'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.461388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207534d06b7742dc8502520a575f35d7', 'request_id': 'req-b5c78b3d-2110-487b-a993-fbb6e02ae097'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.517027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1353/2730] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/users => generated 280 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:36:05.521139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.531718 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cd35a-037e-40a0-b636-1694ac1629c3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.538634 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] The token cache is being invalidate because project 207534d06b7742dc8502520a575f35d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:05.544820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.544820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b5c78b3d-2110-487b-a993-fbb6e02ae097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:05.544820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1378/2731] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:05 2022] DELETE /identity/v3/projects/207534d06b7742dc8502520a575f35d7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:05.546139 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a2138-a81a-4f19-8387-4952780d894a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.546424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a2138-a81a-4f19-8387-4952780d894a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.546699 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a2138-a81a-4f19-8387-4952780d894a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.566083 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a07a2138-a81a-4f19-8387-4952780d894a None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:05.567296 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a07a2138-a81a-4f19-8387-4952780d894a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.590849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.591456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.592074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.592198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.592842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.594442 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.609070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1354/2732] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:36:05.610019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.611830 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.624116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa2937-9312-4b19-ab3a-3b8584c1e793 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.629147 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.629850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.630131 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.630399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.631061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.638002 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1379/2733] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:05.638128 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.639556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.639837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.640140 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.646582 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1355/2734] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:36:05.647727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.657972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615845da-4807-4b44-ae17-7b4c82f65e8f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.661900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:05.662638 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858aec4b-2b10-464c-8ca1-028a65ad522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.663188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.663321 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aec4b-2b10-464c-8ca1-028a65ad522c admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.663539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aec4b-2b10-464c-8ca1-028a65ad522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.663812 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858aec4b-2b10-464c-8ca1-028a65ad522c admin admin] PATH_INFO: `/v3/projects/4da316bb1b7c4ae1b87ee5c235c66a64/users/45f30d765bdd492b9bbac6e50ba697e9/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.671610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1380/2735] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:05.675883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.678301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858aec4b-2b10-464c-8ca1-028a65ad522c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4da316bb1b7c4ae1b87ee5c235c66a64, user_id=45f30d765bdd492b9bbac6e50ba697e9, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.681512 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858aec4b-2b10-464c-8ca1-028a65ad522c admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.685754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b632e71-0fbb-4552-8b00-21363dd22739 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.690582 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.694890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.694890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.694890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.694890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:05.694890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.694890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.706314 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1381/2736] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:36:05.710552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.720697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09242d78-1bf5-47e9-af7a-229736031ba6 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.725306 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.725915 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.726187 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.726452 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.727042 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.728910 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.773083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1356/2737] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:05 2022] PUT /identity/v3/projects/4da316bb1b7c4ae1b87ee5c235c66a64/users/45f30d765bdd492b9bbac6e50ba697e9/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:05.784653 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.784939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.785212 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.811487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:INSERT=6,keystone:DELETE=1,keystone:SELECT=93 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:05.814355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] MFA Rules not processed for user `45f30d765bdd492b9bbac6e50ba697e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:05.815567 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.849057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1382/2738] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/projects => generated 363 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:36:05.853141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.880697 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1357/2739] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:05.916617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:05.918100 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:05.923444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d00cdb-c5fd-4ff4-a4a4-b2722dd9ced7 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:05.928075 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:05.928682 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:05.928958 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:05.929227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:05.929846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:05.931745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:05.976132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46fec7-e5f2-40d8-a42c-1a36d66a5a52 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.023483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.024126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.024402 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.024675 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.042133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1383/2740] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:05 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:06.047395 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.057104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9f76b-7a2c-4f75-a746-de18928b1f44 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.062065 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.062896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.063271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.063549 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.064242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.065833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.075852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1384/2741] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:36:06.079734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.084173 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.085616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.090152 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5655dd-cf57-4187-b1c7-15bb21a0146a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.094833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.095594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1358/2742] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:36:06.096082 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.096371 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.096673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.097335 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.098921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.104347 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:06.110959 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1385/2743] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:36:06.115163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.117947 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.124554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfaaa7a-9bfe-4208-b6da-6ce4cd28d895 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.128678 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ee87e-1c4c-472a-b3c9-ca1b8e2c7b1e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.129481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.130103 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.130445 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.130660 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] PATH_INFO: `/v3/users/45f30d765bdd492b9bbac6e50ba697e9` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.131876 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f30d765bdd492b9bbac6e50ba697e9)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.133719 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.134107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb7a0a-5688-43d4-b52c-e8a39a2516d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.134751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb7a0a-5688-43d4-b52c-e8a39a2516d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.135035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb7a0a-5688-43d4-b52c-e8a39a2516d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.135307 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb7a0a-5688-43d4-b52c-e8a39a2516d2 admin admin] PATH_INFO: `/v3/projects/dbb2df5c8a38484f9ebe337b4fa685a7/users/b9a41e585e6f48309bb94b6cf4814198/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.140030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=100 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:06.150116 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb7a0a-5688-43d4-b52c-e8a39a2516d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb2df5c8a38484f9ebe337b4fa685a7, user_id=b9a41e585e6f48309bb94b6cf4814198, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.152953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb7a0a-5688-43d4-b52c-e8a39a2516d2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.281204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1386/2744] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:06 2022] PUT /identity/v3/projects/dbb2df5c8a38484f9ebe337b4fa685a7/users/b9a41e585e6f48309bb94b6cf4814198/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:06.293420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.293701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.293976 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.321571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] MFA Rules not processed for user `b9a41e585e6f48309bb94b6cf4814198`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:06.322780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:06.386788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1387/2745] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:06.391061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.438851 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f30d765bdd492b9bbac6e50ba697e9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:06.439815 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f30d765bdd492b9bbac6e50ba697e9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:06.440489 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f30d765bdd492b9bbac6e50ba697e9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:06.447473 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f30d765bdd492b9bbac6e50ba697e9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:06.452020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4661f-1be6-4c8e-b1e3-d762be745b0e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.456705 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.457302 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.457578 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.457846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.458458 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.460383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.584487 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-0ee9a8a1-8528-4b73-9697-db816fa344d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f30d765bdd492b9bbac6e50ba697e9'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:06.604291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1359/2746] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:06 2022] DELETE /identity/v3/users/45f30d765bdd492b9bbac6e50ba697e9 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:06.608896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.609182 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.609455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.628544 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:06.629739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:06.631932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1388/2747] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:06 2022] POST /identity/v3/projects => generated 363 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:36:06.638932 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.640950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:06.691468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1360/2748] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:06.694612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:06.701689 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.703392 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:06.708424 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1bb05-8044-4bce-bcc9-620d23933ede admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.713544 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.714228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.714461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.714869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.715502 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.717436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.764149 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f43c1-90d4-47f0-a592-60457954e885 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.768029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.768628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.768904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.769169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.778893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.789102 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.789370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1361/2749] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:36:06.824620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1389/2750] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:06 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:36:06.828599 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.838390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2d7788-93f4-4a71-8d7f-b566d2e4a87d service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.843681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.844312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.844592 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.844862 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.845495 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.847083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:06.858035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1362/2751] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:06.862215 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:06.873365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502aa6c-c0a3-4a6e-adbc-171e9994aa9f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:06.878164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:06.878814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:06.879303 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:06.879717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] PATH_INFO: `/v3/projects/bb209cc569784bbe9da8f95cdb3394a1/users/23a6ccfc578f41cab29db76de82157cf/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:06.895166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb209cc569784bbe9da8f95cdb3394a1, user_id=23a6ccfc578f41cab29db76de82157cf, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:06.897911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:07.001902 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1390/2752] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:06 2022] PUT /identity/v3/projects/bb209cc569784bbe9da8f95cdb3394a1/users/23a6ccfc578f41cab29db76de82157cf/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:07.005941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:07.016040 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b2b1-df58-4f3d-ae7b-12ec2934367b admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:07.028628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:07.072441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd41ea0-c317-4321-87ad-a8d82e737df4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:07.073046 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd41ea0-c317-4321-87ad-a8d82e737df4 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.073327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd41ea0-c317-4321-87ad-a8d82e737df4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.073598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd41ea0-c317-4321-87ad-a8d82e737df4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.074305 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd41ea0-c317-4321-87ad-a8d82e737df4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:07.075890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd41ea0-c317-4321-87ad-a8d82e737df4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:07.086155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1363/2753] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:07.090195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:07.100557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea119f-358f-4577-a5ee-67a903a44883 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:07.105831 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:07.106552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.106834 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.107109 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] PATH_INFO: `/v3/projects/bb209cc569784bbe9da8f95cdb3394a1/users/23a6ccfc578f41cab29db76de82157cf/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.109077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb209cc569784bbe9da8f95cdb3394a1, user_id=23a6ccfc578f41cab29db76de82157cf, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:07.111695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:07.122622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1391/2754] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:07 2022] PUT /identity/v3/projects/bb209cc569784bbe9da8f95cdb3394a1/users/23a6ccfc578f41cab29db76de82157cf/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:07.133207 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.133494 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.133771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.162288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] MFA Rules not processed for user `23a6ccfc578f41cab29db76de82157cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:07.163384 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:07.236299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1364/2755] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:07.248537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:07.250214 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:07.314228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf6351-af77-4a35-9504-8c0c98cc2859 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:07.318141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397aaa7-0e6a-470e-9a2b-f34203e6fff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:07.318738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397aaa7-0e6a-470e-9a2b-f34203e6fff1 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.319010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397aaa7-0e6a-470e-9a2b-f34203e6fff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.319275 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397aaa7-0e6a-470e-9a2b-f34203e6fff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.329436 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397aaa7-0e6a-470e-9a2b-f34203e6fff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:07.330847 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397aaa7-0e6a-470e-9a2b-f34203e6fff1 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:07.340888 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1392/2756] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:07.410711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:07.412408 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:07.490519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c8b72-25e0-42dc-89ba-964165e48eff None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:07.495372 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:07.496546 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.496818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.502932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] PATH_INFO: `/v3/projects/4da316bb1b7c4ae1b87ee5c235c66a64` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.502932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4da316bb1b7c4ae1b87ee5c235c66a64)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:07.502932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:07.616216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:07.624786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4da316bb1b7c4ae1b87ee5c235c66a64', 'request_id': 'req-e282d4f4-a568-4faa-900b-a79fca5bbe60'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:07.631676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4da316bb1b7c4ae1b87ee5c235c66a64', 'request_id': 'req-e282d4f4-a568-4faa-900b-a79fca5bbe60'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:07.638373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4da316bb1b7c4ae1b87ee5c235c66a64', 'request_id': 'req-e282d4f4-a568-4faa-900b-a79fca5bbe60'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:07.727110 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] The token cache is being invalidate because project 4da316bb1b7c4ae1b87ee5c235c66a64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:07.727404 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:07.728087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-e282d4f4-a568-4faa-900b-a79fca5bbe60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:07.729596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1365/2757] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:07 2022] DELETE /identity/v3/projects/4da316bb1b7c4ae1b87ee5c235c66a64 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:07.735071 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.735352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.735619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.755742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:07.756961 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:07.817811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1393/2758] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:07.826287 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.826569 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.826842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.851138 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:07.852674 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:07.861181 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1366/2759] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:07.865466 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:07.875115 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea288e-10a1-46d8-bd13-4be78643643b None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:07.879621 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:07.892244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.892244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.892244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.892244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:07.892244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:07.892244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:07.895246 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1394/2760] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 10 11:36:07.899453 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:07.909438 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a9b5f-1712-4f92-9a05-83948f3dcc4c None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:07.914196 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:07.914790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:07.915066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:07.915338 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:07.915968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:07.917878 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.096744 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1367/2761] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:07 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:08.107327 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.117341 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63629a81-c86e-412c-a283-deaf0e85a971 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.147909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:08.177810 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.178409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.178680 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.178943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.179606 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.181535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.328040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1395/2762] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:08 2022] POST /identity/v3/users => generated 288 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:08.332018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.342531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.342739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a3e6a9-edf7-4e71-8e96-c99a0a97afd2 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.344284 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:08.347355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.347947 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.348252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.348520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.349163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.350759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.356967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bd377-aa7a-403c-87b0-568e309103f1 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.362228 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1368/2763] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:08.362374 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.362495 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.363588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.363588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] PATH_INFO: `/v3/users/b9a41e585e6f48309bb94b6cf4814198` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.364243 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a41e585e6f48309bb94b6cf4814198)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.366314 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.366511 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.377673 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab35807-4493-4980-addc-f456a3b63d79 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.385997 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.386679 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.387024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.387357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.388681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.390433 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.405186 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1369/2764] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:36:08.409439 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.419525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09ef733-3d7c-4681-b7b8-ab63c9b78239 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.424140 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fe400a-c24d-4ba4-95c6-bd6605ae3435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.424750 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe400a-c24d-4ba4-95c6-bd6605ae3435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.425034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe400a-c24d-4ba4-95c6-bd6605ae3435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.425291 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fe400a-c24d-4ba4-95c6-bd6605ae3435 admin admin] PATH_INFO: `/v3/projects/3430c35d599f4b2bbd594fdca325f1ca/users/008fc40d014c4bd186ffdec4a9ba5757/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.441044 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe400a-c24d-4ba4-95c6-bd6605ae3435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3430c35d599f4b2bbd594fdca325f1ca, user_id=008fc40d014c4bd186ffdec4a9ba5757, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.443637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fe400a-c24d-4ba4-95c6-bd6605ae3435 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.527326 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1370/2765] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:08 2022] PUT /identity/v3/projects/3430c35d599f4b2bbd594fdca325f1ca/users/008fc40d014c4bd186ffdec4a9ba5757/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:08.538928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.539221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.539481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.564611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a41e585e6f48309bb94b6cf4814198'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:08.565351 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a41e585e6f48309bb94b6cf4814198'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:08.565971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a41e585e6f48309bb94b6cf4814198'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:08.570162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] MFA Rules not processed for user `008fc40d014c4bd186ffdec4a9ba5757`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:08.571378 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:08.573204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a41e585e6f48309bb94b6cf4814198'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:08.636359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1371/2766] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:08.641313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-9246e231-660e-4068-8880-4f310dd89899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a41e585e6f48309bb94b6cf4814198'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:08.649780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.652259 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:08.660101 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1396/2767] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:08 2022] DELETE /identity/v3/users/b9a41e585e6f48309bb94b6cf4814198 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:08.664701 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.664989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.665264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.685490 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:08.686462 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:08.698033 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:08.709560 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb1a36f-112a-4e83-9a01-25307a96a8ff None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.744803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1397/2768] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:08.755266 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:08.756636 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:08.760379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.761126 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.761409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.761678 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.796837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:08.813529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a304d5-3e0a-4623-b281-34c132f53e10 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:08.817277 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:08.817876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:08.818236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:08.818510 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:08.818621 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.820023 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.827479 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:08.828943 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:08.829066 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1372/2769] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:36:08.839882 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1398/2770] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:36:09.236614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:09.238029 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:09.263508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562cf780-f20b-461c-8084-b5395142c46b service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:09.268132 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:09.268772 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:09.269054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:09.269324 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] PATH_INFO: `/v3/projects/dbb2df5c8a38484f9ebe337b4fa685a7` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:09.270454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb2df5c8a38484f9ebe337b4fa685a7)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:09.272382 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:09.414288 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb2df5c8a38484f9ebe337b4fa685a7', 'request_id': 'req-5f25a334-d13d-431d-8a22-a8be2a43879f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.420423 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb2df5c8a38484f9ebe337b4fa685a7', 'request_id': 'req-5f25a334-d13d-431d-8a22-a8be2a43879f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.425991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb2df5c8a38484f9ebe337b4fa685a7', 'request_id': 'req-5f25a334-d13d-431d-8a22-a8be2a43879f'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.499573 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] The token cache is being invalidate because project dbb2df5c8a38484f9ebe337b4fa685a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:09.499864 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.500529 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.502014 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1373/2771] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:09 2022] DELETE /identity/v3/projects/dbb2df5c8a38484f9ebe337b4fa685a7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:09.508670 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:09.516593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222f058-cc63-4771-bfd4-c64efcb8c47a service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:09.574218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:09.574725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:09.575581 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:09.575849 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] PATH_INFO: `/v3/users/23a6ccfc578f41cab29db76de82157cf` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:09.577095 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a6ccfc578f41cab29db76de82157cf)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:09.579128 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:09.751865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a6ccfc578f41cab29db76de82157cf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.752532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a6ccfc578f41cab29db76de82157cf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.753126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a6ccfc578f41cab29db76de82157cf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.759787 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a6ccfc578f41cab29db76de82157cf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.818136 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-b122037c-ce31-4dee-b444-a5cc17f64e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a6ccfc578f41cab29db76de82157cf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:09.836574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1399/2772] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:09 2022] DELETE /identity/v3/users/23a6ccfc578f41cab29db76de82157cf => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:10.277253 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:10.279308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:10.292565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25a334-d13d-431d-8a22-a8be2a43879f admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:10.327752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:10.350554 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:10.351185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:10.351469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:10.351746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] PATH_INFO: `/v3/projects/bb209cc569784bbe9da8f95cdb3394a1` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:10.352904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb209cc569784bbe9da8f95cdb3394a1)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:10.354803 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:10.453709 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb209cc569784bbe9da8f95cdb3394a1', 'request_id': 'req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:10.461283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb209cc569784bbe9da8f95cdb3394a1', 'request_id': 'req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:10.468914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb209cc569784bbe9da8f95cdb3394a1', 'request_id': 'req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:10.561189 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] The token cache is being invalidate because project bb209cc569784bbe9da8f95cdb3394a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:10.561480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:10.562118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c80ea1ff-ded4-4a17-abf4-8af9001f4fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:10.563606 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1374/2773] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:10 2022] DELETE /identity/v3/projects/bb209cc569784bbe9da8f95cdb3394a1 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:10.568529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:10.568992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:10.569288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:10.598010 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:10.599270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:10.658642 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:10.669337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1400/2774] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:10.677421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:10.677705 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:10.677976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:10.697938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:10.699139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:10.708077 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1375/2775] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:10.715388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:10.725786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca0894-9212-4bb3-b1f0-6d0d93e5aef2 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:10.730446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:10.731070 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:10.731729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:10.731729 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:10.732365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:10.732667 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:10.734377 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:10.746072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1401/2776] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:10.752322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:10.759891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce194d-fbe0-40c7-8af8-43d8f17bdb2b None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:10.765279 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:10.765872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:10.766152 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:10.766426 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:10.767050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:10.768985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:10.917957 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1376/2777] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:10 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:10.921951 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:10.931862 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e48258-b734-4ee3-a88a-6a4c31d0e20a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.000588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.001199 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.001480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.001750 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.002426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.004356 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.076454 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1402/2778] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:10 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:11.080906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.092159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70572a5-fb87-4fb2-a462-2fdb03011941 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.096980 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.097586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.097865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.098139 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.098826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.101462 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.112184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1377/2779] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:11.116204 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.128661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cf053-167f-4efa-a376-c10f1397c5d9 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.133610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb9583-fc1f-4393-bd51-9805dfcbb922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.133985 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb9583-fc1f-4393-bd51-9805dfcbb922 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.134263 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb9583-fc1f-4393-bd51-9805dfcbb922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.134533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb9583-fc1f-4393-bd51-9805dfcbb922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.135164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb9583-fc1f-4393-bd51-9805dfcbb922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.136807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb9583-fc1f-4393-bd51-9805dfcbb922 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.148500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1403/2780] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:11.152539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.163738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaaf0b2-2046-4833-bd81-b51b0bac4b55 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.168525 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.169867 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.170188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.170464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] PATH_INFO: `/v3/projects/2124f5658e43459cb378da463bf675f6/users/39b5b7f6b5e3450d85c31296b3af951f/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.185304 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2124f5658e43459cb378da463bf675f6, user_id=39b5b7f6b5e3450d85c31296b3af951f, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.187983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.252619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1378/2781] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:11 2022] PUT /identity/v3/projects/2124f5658e43459cb378da463bf675f6/users/39b5b7f6b5e3450d85c31296b3af951f/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:11.264090 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.264375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.264649 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.292521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] MFA Rules not processed for user `39b5b7f6b5e3450d85c31296b3af951f`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:11.294368 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:11.375399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1404/2782] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:11.383889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.444187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd636db2-ad86-4a4e-ba30-5f5ea8b6566a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.448831 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.449420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.449692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.450054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.450919 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.452843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.462022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:11.539250 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1379/2783] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:11 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:11.543264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.554574 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1df5-35bb-493b-a7c2-37c1c20f1c9e None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.601691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:11.610702 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.611346 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.611623 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.611891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.612543 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.614443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.710772 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1405/2784] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:11 2022] POST /identity/v3/users => generated 288 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:11.714672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.725661 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a89a57-d6fa-4819-a64c-ba057f67acc2 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.730206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.730887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.731652 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.732524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.733167 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.734749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.745931 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1380/2785] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:11.755148 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.765403 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4d268-94e2-46f4-af92-d55c366f89c4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:11.770144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:11.770788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:11.771072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:11.771339 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] PATH_INFO: `/v3/projects/746cd3f2580a4e95ab84e36565a9f3dd/users/dd5ec705063e4df9923dba3ab3a3722a/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:11.786587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] RBAC: Authorizing `identity:create_grant(project_id=746cd3f2580a4e95ab84e36565a9f3dd, user_id=dd5ec705063e4df9923dba3ab3a3722a, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:11.789594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:11.927789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1406/2786] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:11 2022] PUT /identity/v3/projects/746cd3f2580a4e95ab84e36565a9f3dd/users/dd5ec705063e4df9923dba3ab3a3722a/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:11.931775 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:11.942935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede00f-24a6-49d8-9c5b-8194ce430708 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:12.005695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239b37e-1e38-4219-adba-490e5bace257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:12.006468 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b37e-1e38-4219-adba-490e5bace257 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:12.006748 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b37e-1e38-4219-adba-490e5bace257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:12.007019 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b37e-1e38-4219-adba-490e5bace257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:12.007658 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b37e-1e38-4219-adba-490e5bace257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:12.009389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b37e-1e38-4219-adba-490e5bace257 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:12.020451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1381/2787] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:12.026040 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:12.036405 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b3ec70-5d28-4130-b832-a51a0806278c admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:12.040992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:12.041695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:12.041974 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:12.042249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] PATH_INFO: `/v3/projects/746cd3f2580a4e95ab84e36565a9f3dd/users/dd5ec705063e4df9923dba3ab3a3722a/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:12.044385 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=746cd3f2580a4e95ab84e36565a9f3dd, user_id=dd5ec705063e4df9923dba3ab3a3722a, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:12.047098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:12.057775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1407/2788] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:12 2022] PUT /identity/v3/projects/746cd3f2580a4e95ab84e36565a9f3dd/users/dd5ec705063e4df9923dba3ab3a3722a/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:12.068778 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:12.069057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:12.069325 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:12.105627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] MFA Rules not processed for user `dd5ec705063e4df9923dba3ab3a3722a`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:12.107458 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:12.175805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1382/2789] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:12.189242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:12.190953 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:12.248717 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b259495-1d1e-4b79-a86f-07ad57b001ab admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:12.252558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:12.253155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:12.253437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:12.253708 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:12.263195 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:12.264662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:12.274802 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1408/2790] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:36:12.468922 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:12.478871 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcac40a-8785-47d0-b769-1a1ccc8716e3 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:12.483172 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aac63-1f74-4604-ac71-ac4bd7af3b6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:12.483768 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aac63-1f74-4604-ac71-ac4bd7af3b6e service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:12.484078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aac63-1f74-4604-ac71-ac4bd7af3b6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:12.484352 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aac63-1f74-4604-ac71-ac4bd7af3b6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:12.548870 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aac63-1f74-4604-ac71-ac4bd7af3b6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:12.550379 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aac63-1f74-4604-ac71-ac4bd7af3b6e service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:12.553902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:12.559975 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1383/2791] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:18.187736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:18.249324 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356efa4-5026-47e2-a27d-ad956cd933fd service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:18.253837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:18.254464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:18.254741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:18.255011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] PATH_INFO: `/v3/users/39b5b7f6b5e3450d85c31296b3af951f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:18.256150 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] RBAC: Authorizing `identity:delete_user(user_id=39b5b7f6b5e3450d85c31296b3af951f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:18.258047 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:18.513627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39b5b7f6b5e3450d85c31296b3af951f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:18.514272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39b5b7f6b5e3450d85c31296b3af951f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:18.514871 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39b5b7f6b5e3450d85c31296b3af951f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:18.516833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:18.521933 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39b5b7f6b5e3450d85c31296b3af951f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:18.629727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39b5b7f6b5e3450d85c31296b3af951f'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:18.647881 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1409/2792] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:18 2022] DELETE /identity/v3/users/39b5b7f6b5e3450d85c31296b3af951f => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:18.656183 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:18.656672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:18.656781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:18.675441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:18.676728 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:18.735421 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1384/2793] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:18.745060 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:18.746464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:18.807954 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d714089-8d1d-47f1-a5b0-8bd56042320e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:18.812353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:18.825485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:18.825485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:18.825485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:18.825485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:18.825485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:18.834426 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1410/2794] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:19.220176 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:19.221686 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:19.234395 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93855a1a-65f6-4cce-803f-730b95d84ba1 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:19.239320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:19.239979 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:19.240299 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:19.240637 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] PATH_INFO: `/v3/projects/2124f5658e43459cb378da463bf675f6` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:19.241791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2124f5658e43459cb378da463bf675f6)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:19.243703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:19.341358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2124f5658e43459cb378da463bf675f6', 'request_id': 'req-ef66946f-3e63-43a3-bc3e-c1d0ac744757'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.348668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2124f5658e43459cb378da463bf675f6', 'request_id': 'req-ef66946f-3e63-43a3-bc3e-c1d0ac744757'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.354340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2124f5658e43459cb378da463bf675f6', 'request_id': 'req-ef66946f-3e63-43a3-bc3e-c1d0ac744757'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.550229 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] The token cache is being invalidate because project 2124f5658e43459cb378da463bf675f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:19.550508 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.551057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.552588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1385/2795] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:19 2022] DELETE /identity/v3/projects/2124f5658e43459cb378da463bf675f6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:19.556533 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:19.566309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cb37c-bba6-4d20-ae2e-200203ec75ea service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:19.622200 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:19.622863 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:19.623141 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:19.623409 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] PATH_INFO: `/v3/users/dd5ec705063e4df9923dba3ab3a3722a` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:19.624552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5ec705063e4df9923dba3ab3a3722a)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:19.626443 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:19.798144 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5ec705063e4df9923dba3ab3a3722a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.798783 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5ec705063e4df9923dba3ab3a3722a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.799458 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5ec705063e4df9923dba3ab3a3722a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.806718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5ec705063e4df9923dba3ab3a3722a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.891821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-3d3ca131-c71c-49b5-8d68-6a420d8f65d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5ec705063e4df9923dba3ab3a3722a'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:19.899674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:19.910756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1411/2796] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:19 2022] DELETE /identity/v3/users/dd5ec705063e4df9923dba3ab3a3722a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:20.340780 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:20.342331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:20.362350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef66946f-3e63-43a3-bc3e-c1d0ac744757 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:20.419900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:20.420555 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:20.420829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:20.421097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] PATH_INFO: `/v3/projects/746cd3f2580a4e95ab84e36565a9f3dd` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:20.422358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=746cd3f2580a4e95ab84e36565a9f3dd)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:20.424274 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:20.448268 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:20.533144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746cd3f2580a4e95ab84e36565a9f3dd', 'request_id': 'req-4f5daa34-e385-486a-9b49-d7df452f3567'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:20.539315 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746cd3f2580a4e95ab84e36565a9f3dd', 'request_id': 'req-4f5daa34-e385-486a-9b49-d7df452f3567'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:20.544766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746cd3f2580a4e95ab84e36565a9f3dd', 'request_id': 'req-4f5daa34-e385-486a-9b49-d7df452f3567'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:20.616357 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] The token cache is being invalidate because project 746cd3f2580a4e95ab84e36565a9f3dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:20.616640 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:20.617216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4f5daa34-e385-486a-9b49-d7df452f3567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:20.618761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1386/2797] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:20 2022] DELETE /identity/v3/projects/746cd3f2580a4e95ab84e36565a9f3dd => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:20.623886 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:20.624410 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:20.624524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:20.644299 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:20.645505 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:20.710415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1412/2798] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:20.719672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:20.720067 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:20.720420 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:20.739850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:20.741093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:20.749616 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1387/2799] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:20.756310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:20.766770 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c8e42-4120-4ac3-82da-1c77528531e7 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:20.771313 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:20.771898 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:20.772311 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:20.772584 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:20.773219 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:20.773517 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:20.775096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:20.786674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1413/2800] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:20.790893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:20.800075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e95f27-efec-44bb-afe4-ca02382c6c37 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:20.805054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:20.805365 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:20.805644 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:20.805917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:20.806539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:20.808481 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:20.935045 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1388/2801] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:20 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:20.939499 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:20.949406 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294b771-3ab5-4b87-a5c9-8bf58248071a admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:21.006456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:21.007056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:21.007437 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:21.007654 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:21.008340 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:21.010232 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:21.110662 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1414/2802] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:20 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:21.115158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:21.129158 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e474b39-9c5b-489a-b31a-977d14928c47 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:21.133868 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:21.134827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:21.135108 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:21.135376 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:21.136078 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:21.138521 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:21.148657 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1389/2803] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:21.152895 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:21.163205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb3519-a755-4ca4-b31c-4fd687530517 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:21.167892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48699f9a-91f9-4de2-98fa-39bc19f34518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:21.169745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699f9a-91f9-4de2-98fa-39bc19f34518 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:21.169745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699f9a-91f9-4de2-98fa-39bc19f34518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:21.169745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48699f9a-91f9-4de2-98fa-39bc19f34518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:21.170205 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48699f9a-91f9-4de2-98fa-39bc19f34518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:21.171777 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48699f9a-91f9-4de2-98fa-39bc19f34518 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:21.182248 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1415/2804] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:21.187009 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:21.197080 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b190b4e-423f-43bc-9d81-4fe20aaaaa13 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:21.201880 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:21.207333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:21.207333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:21.207333 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] PATH_INFO: `/v3/projects/ba59b747bb824fe89766600ee0f7504d/users/f625c8881f1e41daacc79e5a8589c3fe/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:21.218221 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba59b747bb824fe89766600ee0f7504d, user_id=f625c8881f1e41daacc79e5a8589c3fe, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:21.220903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:21.315385 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1390/2805] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:21 2022] PUT /identity/v3/projects/ba59b747bb824fe89766600ee0f7504d/users/f625c8881f1e41daacc79e5a8589c3fe/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:36:21.326996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:21.327274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:21.327540 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:21.354792 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] MFA Rules not processed for user `f625c8881f1e41daacc79e5a8589c3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:21.356608 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:21.402096 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:21.423398 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1416/2806] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:21.435393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:21.440409 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:21.515444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d5bab-e893-4153-a956-58c46b988495 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:21.519191 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7feac8-47e8-4971-98aa-355f8204ac36 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a95ab030c9940c9946c64c82567773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:21.519792 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7feac8-47e8-4971-98aa-355f8204ac36 service glance] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:21.520095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7feac8-47e8-4971-98aa-355f8204ac36 service glance] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:21.520371 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7feac8-47e8-4971-98aa-355f8204ac36 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:21.529213 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7feac8-47e8-4971-98aa-355f8204ac36 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:21.530631 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7feac8-47e8-4971-98aa-355f8204ac36 service glance] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:21.539350 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1391/2807] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:31.535475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=77,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:31.600842 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=7 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:39.254824 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:39.256262 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:39.320552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28e0e4-774c-498c-98e1-55cc9c159ca6 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:39.325098 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:39.325727 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:39.326003 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:39.326272 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] PATH_INFO: `/v3/users/008fc40d014c4bd186ffdec4a9ba5757` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:39.327421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008fc40d014c4bd186ffdec4a9ba5757)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:39.329359 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:39.586051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008fc40d014c4bd186ffdec4a9ba5757'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:39.586696 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008fc40d014c4bd186ffdec4a9ba5757'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:39.587321 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008fc40d014c4bd186ffdec4a9ba5757'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:39.595357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008fc40d014c4bd186ffdec4a9ba5757'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:39.675051 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008fc40d014c4bd186ffdec4a9ba5757'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:39.692294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1417/2808] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:39 2022] DELETE /identity/v3/users/008fc40d014c4bd186ffdec4a9ba5757 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:39.696900 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:39.697359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:39.697472 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:39.716635 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:39.717828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:39.774877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1392/2809] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:39.785353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:39.786736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:39.845308 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d538e7d-65ca-486e-92a6-7eb05a95cf91 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:39.848901 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1722d755-93d2-4aaa-9e21-74e5530ec12f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:39.853603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722d755-93d2-4aaa-9e21-74e5530ec12f service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:39.853603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722d755-93d2-4aaa-9e21-74e5530ec12f service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:39.853603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722d755-93d2-4aaa-9e21-74e5530ec12f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:39.859011 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722d755-93d2-4aaa-9e21-74e5530ec12f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:39.860399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1722d755-93d2-4aaa-9e21-74e5530ec12f service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:39.869579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1418/2810] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:40.311400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:40.313368 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:40.326879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c6e7e-7eb3-42b0-be57-efad63c849fc None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:40.333076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:40.333724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:40.334034 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:40.334312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] PATH_INFO: `/v3/projects/3430c35d599f4b2bbd594fdca325f1ca` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:40.335451 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3430c35d599f4b2bbd594fdca325f1ca)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:40.338355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:40.459085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3430c35d599f4b2bbd594fdca325f1ca', 'request_id': 'req-c7f7150e-63ff-443c-a730-b30346ddcd1d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:40.468545 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3430c35d599f4b2bbd594fdca325f1ca', 'request_id': 'req-c7f7150e-63ff-443c-a730-b30346ddcd1d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:40.474734 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3430c35d599f4b2bbd594fdca325f1ca', 'request_id': 'req-c7f7150e-63ff-443c-a730-b30346ddcd1d'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:40.563791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] The token cache is being invalidate because project 3430c35d599f4b2bbd594fdca325f1ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:40.563791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:40.563791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7f7150e-63ff-443c-a730-b30346ddcd1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:40.564185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1393/2811] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:40 2022] DELETE /identity/v3/projects/3430c35d599f4b2bbd594fdca325f1ca => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:40.571053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691e060-820e-4130-b57c-3a364a027294 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:40.571577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691e060-820e-4130-b57c-3a364a027294 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:40.571691 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f691e060-820e-4130-b57c-3a364a027294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:40.597445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f691e060-820e-4130-b57c-3a364a027294 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:40.598738 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f691e060-820e-4130-b57c-3a364a027294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:40.619908 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:40.668785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1419/2812] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:40.677524 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:40.677807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:40.678082 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:40.710953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:40.712201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:40.723245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1394/2813] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:40.727919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691e060-820e-4130-b57c-3a364a027294 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:40.742107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f691e060-820e-4130-b57c-3a364a027294 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:40.750587 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:40.751362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:40.751760 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:40.752191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:40.753133 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:40.753576 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:40.756077 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:40.768852 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1420/2814] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:40.773427 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:40.782766 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd7118-650a-476f-9085-e3b46f620a77 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:40.787320 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:40.787902 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:40.788208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:40.788475 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:40.789093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:40.790972 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:40.957364 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1395/2815] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:40 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:36:40.961532 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:40.972170 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e8398-fcc4-4f76-bb43-97088bce43a2 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:41.026552 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:41.027227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:41.027507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:41.027776 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:41.028415 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:41.030291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:41.109536 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1421/2816] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:40 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:41.113903 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:41.123964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251cd6ba-2591-4544-b1d0-05608df270c9 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:41.130060 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:41.130645 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:41.130938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:41.131205 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:41.131893 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:41.133507 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:41.144184 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1396/2817] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:41.148212 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:41.158817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52619bcb-28c9-4347-bbd1-dadf961858df admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:41.163396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f909a0-514c-4a71-8abe-67445602023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:41.163981 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f909a0-514c-4a71-8abe-67445602023e admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:41.164291 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f909a0-514c-4a71-8abe-67445602023e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:41.164558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f909a0-514c-4a71-8abe-67445602023e admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:41.165183 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f909a0-514c-4a71-8abe-67445602023e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:41.166744 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f909a0-514c-4a71-8abe-67445602023e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:41.178330 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1422/2818] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:41.182312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:41.193283 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17933f3-3841-4179-85c6-d9adb46249cd admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:41.197845 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:41.198506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:41.198786 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:41.199055 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] PATH_INFO: `/v3/projects/e8775e002af14ee2ac9d6601a6872dfa/users/63e9c06c2c4f4caabf6ecd67cece94f4/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:41.217690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8775e002af14ee2ac9d6601a6872dfa, user_id=63e9c06c2c4f4caabf6ecd67cece94f4, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:41.220739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:41.308179 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1397/2819] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:41 2022] PUT /identity/v3/projects/e8775e002af14ee2ac9d6601a6872dfa/users/63e9c06c2c4f4caabf6ecd67cece94f4/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:41.314309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b9dc5-139e-43d2-a01f-f8b30fb9bef6 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:41.317751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b9dc5-139e-43d2-a01f-f8b30fb9bef6 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:41.317751 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b9dc5-139e-43d2-a01f-f8b30fb9bef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:41.333422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-397b9dc5-139e-43d2-a01f-f8b30fb9bef6 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:41.334603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-397b9dc5-139e-43d2-a01f-f8b30fb9bef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:41.392971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1423/2820] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:41.402517 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:41.404197 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:41.426694 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:41.461499 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f26a8-d408-4026-b2f5-7d63108299d1 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:41.465894 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:41.466483 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:41.466758 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:41.467027 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:41.476713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:41.478168 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:41.488198 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1398/2821] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 10 11:36:46.716664 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:46.717593 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:46.717788 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:46.750927 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] MFA Rules not processed for user `63e9c06c2c4f4caabf6ecd67cece94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:46.752896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:46.775127 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:46.833522 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1424/2822] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:46.837826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:46.849951 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263cfd-f768-418a-8dd1-c5abd93c35b1 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:46.856572 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:46.857617 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:46.858339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:46.858805 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:46.861650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:46.865906 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.010284 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1399/2823] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:46 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:47.013655 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:47.026617 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d313c-0c0b-4bde-8166-858e1dd92e6d None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:47.096524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:47.097166 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:47.097445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:47.097713 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:47.098352 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:47.100725 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.231069 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1425/2824] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:47 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:47.235185 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:47.245699 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ac8e4-ac1e-407c-a293-7790fa6bcf39 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:47.250850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:47.251469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:47.251746 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:47.252049 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:47.252688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:47.254258 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.265238 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1400/2825] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:47.270811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:47.281741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b70e6ed-dcba-4530-9da8-c0a6a7d8d1ca admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:47.286433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:47.291175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:47.291175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:47.291175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] PATH_INFO: `/v3/projects/b5c98518cadb400a8fe2ce233d32257a/users/b336e6b2105e48df8c6fa98baf485c20/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:47.303615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c98518cadb400a8fe2ce233d32257a, user_id=b336e6b2105e48df8c6fa98baf485c20, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:47.306270 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.384896 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1426/2826] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:47 2022] PUT /identity/v3/projects/b5c98518cadb400a8fe2ce233d32257a/users/b336e6b2105e48df8c6fa98baf485c20/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:47.389095 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:47.399169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c8e4eb-8c1f-4c1c-97ad-c6cb294ff087 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:47.455707 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:47.456353 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:47.456677 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:47.456950 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:47.457598 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:47.459386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.470188 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1401/2827] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:47.474467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:47.486952 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c685d7-5bc8-43ed-b528-661d6a37e63f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:47.491700 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:47.492416 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:47.492698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:47.492969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] PATH_INFO: `/v3/projects/b5c98518cadb400a8fe2ce233d32257a/users/b336e6b2105e48df8c6fa98baf485c20/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:47.495072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c98518cadb400a8fe2ce233d32257a, user_id=b336e6b2105e48df8c6fa98baf485c20, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:47.497825 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.509698 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1427/2828] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:47 2022] PUT /identity/v3/projects/b5c98518cadb400a8fe2ce233d32257a/users/b336e6b2105e48df8c6fa98baf485c20/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:47.754516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:47.756267 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:47.880121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830ffd8-7577-4fa4-aeac-5ba94b4514af admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:47.914879 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:47.951093 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:47.951738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:47.952047 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:47.952329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:47.954301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:47.966424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:47.968057 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c6e4c-270c-4dd6-9fd5-e7b66d076660 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:47.979104 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1402/2829] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:49.498269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:49.499989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:49.574573 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b2d54-1f98-4415-9e3e-9bef73afb2b4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:49.580811 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:49.582143 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:49.582755 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:49.583193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] PATH_INFO: `/v3/users/f625c8881f1e41daacc79e5a8589c3fe` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:49.585922 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f625c8881f1e41daacc79e5a8589c3fe)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:49.589014 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:49.603861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:49.797878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f625c8881f1e41daacc79e5a8589c3fe'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:49.798789 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f625c8881f1e41daacc79e5a8589c3fe'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:49.799653 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f625c8881f1e41daacc79e5a8589c3fe'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:49.808966 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f625c8881f1e41daacc79e5a8589c3fe'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:49.877610 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f625c8881f1e41daacc79e5a8589c3fe'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:49.907521 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1428/2830] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:36:49 2022] DELETE /identity/v3/users/f625c8881f1e41daacc79e5a8589c3fe => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:49.911853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798e18f-191b-495f-8e37-6740244130cc None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:49.912399 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798e18f-191b-495f-8e37-6740244130cc None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:49.912541 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798e18f-191b-495f-8e37-6740244130cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:49.950650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-5798e18f-191b-495f-8e37-6740244130cc None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:49.952469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5798e18f-191b-495f-8e37-6740244130cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:50.036190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1403/2831] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:50.070650 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:50.073556 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:50.143391 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe48b5-1c7f-41bb-a4ba-0e43e032aec8 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:50.153163 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138bcbc-351d-4435-a271-9537d0afca57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:50.153803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138bcbc-351d-4435-a271-9537d0afca57 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:50.154078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138bcbc-351d-4435-a271-9537d0afca57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:50.154386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138bcbc-351d-4435-a271-9537d0afca57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:50.165383 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138bcbc-351d-4435-a271-9537d0afca57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:50.166745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138bcbc-351d-4435-a271-9537d0afca57 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:50.179741 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1429/2832] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:50.964159 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798e18f-191b-495f-8e37-6740244130cc None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:50.965672 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5798e18f-191b-495f-8e37-6740244130cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:50.979990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798e18f-191b-495f-8e37-6740244130cc None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:50.987455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:50.989585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:50.990192 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:50.993890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] PATH_INFO: `/v3/projects/ba59b747bb824fe89766600ee0f7504d` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:50.993890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba59b747bb824fe89766600ee0f7504d)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:50.997216 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:51.145032 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba59b747bb824fe89766600ee0f7504d', 'request_id': 'req-c7231cfc-3e47-4d79-958a-edb616c74af1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:51.155017 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba59b747bb824fe89766600ee0f7504d', 'request_id': 'req-c7231cfc-3e47-4d79-958a-edb616c74af1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:51.166820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba59b747bb824fe89766600ee0f7504d', 'request_id': 'req-c7231cfc-3e47-4d79-958a-edb616c74af1'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:51.276977 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] The token cache is being invalidate because project ba59b747bb824fe89766600ee0f7504d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:36:51.277327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:51.279030 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-c7231cfc-3e47-4d79-958a-edb616c74af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:36:51.284075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1404/2833] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:36:50 2022] DELETE /identity/v3/projects/ba59b747bb824fe89766600ee0f7504d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:51.287941 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.288480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.288480 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.310244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:51.311226 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:51.402567 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1430/2834] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:51.410774 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.411062 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.411739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.436839 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:51.439538 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:51.449121 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1405/2835] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:51.453465 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:51.465267 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf400cb-fe1b-4144-8af4-7a7020779241 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:51.470622 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:51.471231 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.471513 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.471785 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.472618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:36:51.472968 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:51.474589 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:51.488957 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1431/2836] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:36:51.492990 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:51.507051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b503c11d-b3d3-4d95-b9e1-1963f5e73118 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:51.512208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:51.516793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.516793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.516793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.516793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:51.516793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:51.632292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1406/2837] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:36:51 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 10 11:36:51.633399 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:51.645357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc4f65-28e2-4588-a3b2-c0e39c70d0bc admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:51.667742 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:51.720856 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:51.722225 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.722579 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.723278 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.724822 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:51.726814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:51.811494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1432/2838] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:36:51 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:36:51.817693 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:51.830825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bece6e2-bb55-4ba7-b394-3c675650f61d admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:51.836281 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:51.837359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.837359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.837615 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.838190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:51.840242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:51.853300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1407/2839] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:51.857671 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:51.868413 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a60921-1718-4e56-823c-cb361c315eb0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:51.873998 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87717a1-8a00-4bca-8c6d-78425f757e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:51.874618 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87717a1-8a00-4bca-8c6d-78425f757e92 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.875640 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87717a1-8a00-4bca-8c6d-78425f757e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.875919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87717a1-8a00-4bca-8c6d-78425f757e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.876603 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87717a1-8a00-4bca-8c6d-78425f757e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:51.878213 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87717a1-8a00-4bca-8c6d-78425f757e92 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:51.891609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1433/2840] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:36:51.898101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:51.903520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:36:51.909061 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25d2b2-400e-494a-9d74-2861c788b6ef admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:51.913921 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:51.914588 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:51.914869 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:51.915153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] PATH_INFO: `/v3/projects/90c7d420689c4ae8be418a17b86eb164/users/fe590b43df374957b4cbd0438ca54445/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:51.931695 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90c7d420689c4ae8be418a17b86eb164, user_id=fe590b43df374957b4cbd0438ca54445, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:51.934828 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:52.024194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1408/2841] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:36:51 2022] PUT /identity/v3/projects/90c7d420689c4ae8be418a17b86eb164/users/fe590b43df374957b4cbd0438ca54445/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:36:52.025950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf31eb-a50a-4d5e-baf3-6cd5a478b134 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:52.026237 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf31eb-a50a-4d5e-baf3-6cd5a478b134 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:52.026509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf31eb-a50a-4d5e-baf3-6cd5a478b134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:52.057155 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-1eaf31eb-a50a-4d5e-baf3-6cd5a478b134 None None] MFA Rules not processed for user `fe590b43df374957b4cbd0438ca54445`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:52.059236 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1eaf31eb-a50a-4d5e-baf3-6cd5a478b134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:52.153529 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1434/2842] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:36:52.170265 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:52.173505 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:52.245002 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3fa90-23f8-4a9e-8e7d-2db8d547a4c3 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:52.249039 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:52.250708 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:52.251094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:52.251452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:52.261240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:52.262703 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:52.283860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1409/2843] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:36:54.105827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:54.106343 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:54.106459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:54.136032 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] MFA Rules not processed for user `b336e6b2105e48df8c6fa98baf485c20`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:36:54.137673 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:54.207420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1435/2844] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:36:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:36:54.222632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:36:54.224236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:54.310136 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c13bf4-f0f4-4f8e-be77-3b6a02f0fac1 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:36:54.316824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:36:54.317432 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:36:54.317711 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:36:54.317982 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:36:54.319759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:36:54.379301 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:36:54.380939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730a4f2-31a7-43ff-8645-54d27ac5d77b service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:36:54.390935 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1410/2845] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:36:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:02.539703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:02.554331 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:02.607044 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240de1-854f-4cd7-a7e2-306c06acc91a None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:02.612520 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:02.613156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:02.613433 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:02.613703 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] PATH_INFO: `/v3/users/fe590b43df374957b4cbd0438ca54445` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:02.614827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe590b43df374957b4cbd0438ca54445)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:02.616883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:02.798310 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe590b43df374957b4cbd0438ca54445'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:02.798878 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe590b43df374957b4cbd0438ca54445'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:02.799550 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe590b43df374957b4cbd0438ca54445'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:02.807254 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe590b43df374957b4cbd0438ca54445'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:02.886615 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe590b43df374957b4cbd0438ca54445'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:02.905088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1436/2846] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:37:02 2022] DELETE /identity/v3/users/fe590b43df374957b4cbd0438ca54445 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:02.910201 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:02.910667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:02.910667 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:02.931842 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:02.933072 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:02.945109 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:02.993630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1411/2847] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:03.004743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:03.006171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:03.069290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09a211-aaec-4788-b62f-7564aecb4580 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:03.072992 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a408548-c8b3-4b07-9b33-2a5b73ad6745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:03.073596 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a408548-c8b3-4b07-9b33-2a5b73ad6745 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:03.073876 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a408548-c8b3-4b07-9b33-2a5b73ad6745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:03.074149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a408548-c8b3-4b07-9b33-2a5b73ad6745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:03.083365 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a408548-c8b3-4b07-9b33-2a5b73ad6745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:03.084884 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a408548-c8b3-4b07-9b33-2a5b73ad6745 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:03.095294 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1437/2848] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:03.500231 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:03.501777 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:03.514018 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644c1bb-51cd-4008-b598-baffa60d24ea None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:03.520456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:03.520456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:03.520456 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:03.520843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] PATH_INFO: `/v3/projects/90c7d420689c4ae8be418a17b86eb164` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:03.521843 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90c7d420689c4ae8be418a17b86eb164)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:03.523773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:03.620749 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90c7d420689c4ae8be418a17b86eb164', 'request_id': 'req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:03.627085 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90c7d420689c4ae8be418a17b86eb164', 'request_id': 'req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:03.632680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90c7d420689c4ae8be418a17b86eb164', 'request_id': 'req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:03.704035 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] The token cache is being invalidate because project 90c7d420689c4ae8be418a17b86eb164 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:37:03.704337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:03.705004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6b1405e8-0e87-4ea4-9c76-ef2f019f3798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:03.706519 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1412/2849] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:37:03 2022] DELETE /identity/v3/projects/90c7d420689c4ae8be418a17b86eb164 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:03.711375 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:03.711843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:03.711972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:03.733602 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:03.735381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:03.764572 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:03.794396 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1438/2850] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:03.802872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:03.803156 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:03.803431 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:03.823865 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:03.825119 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:03.834097 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1413/2851] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:03.838318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:03.849661 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17e26-027b-4bff-88b4-419c866dc4b0 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:03.854245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:03.854853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:03.855228 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:03.855645 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:03.856344 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:37:03.856697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:03.858362 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:03.870444 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1439/2852] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:37:03.877614 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:03.884855 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f3fce-3cd7-43df-9d41-1f51d2cdafcf None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:03.889630 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:03.890225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:03.890501 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:03.890769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:03.892076 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:03.893968 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:03.985974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1414/2853] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:37:03 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:37:03.990191 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.001563 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c50a9-1871-41e5-9383-4a3d19a6ff1e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.057161 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.057756 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.058072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.058337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.058982 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.060919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.138827 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1440/2854] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:37:03 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:37:04.146236 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.157246 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab92ae90-b05e-4c07-b690-3d51825f37b4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.162386 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.163001 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.163273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.163539 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.164208 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.165820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.176596 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1415/2855] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:37:04.180688 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.190917 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7afb19c-3eb5-4bda-af6a-04e9b6ef5884 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.197076 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa40fc8-7f3f-468d-a716-790ca6db53f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.197695 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40fc8-7f3f-468d-a716-790ca6db53f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.197972 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40fc8-7f3f-468d-a716-790ca6db53f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.198242 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa40fc8-7f3f-468d-a716-790ca6db53f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.198877 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40fc8-7f3f-468d-a716-790ca6db53f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.200500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa40fc8-7f3f-468d-a716-790ca6db53f0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.212524 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1441/2856] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:37:04.221737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.236852 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477485e-ad19-40a6-b55d-650c208662a4 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.241628 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.242300 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.246858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.246858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] PATH_INFO: `/v3/projects/a432c036587d4010a926f3b3cb86911d/users/1307001c86e84466b939f872f9385516/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.257863 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a432c036587d4010a926f3b3cb86911d, user_id=1307001c86e84466b939f872f9385516, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.260548 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.344904 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1416/2857] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:37:04 2022] PUT /identity/v3/projects/a432c036587d4010a926f3b3cb86911d/users/1307001c86e84466b939f872f9385516/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:04.355887 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.356245 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.356523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.385156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] MFA Rules not processed for user `1307001c86e84466b939f872f9385516`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:04.386384 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:04.450435 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1442/2858] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:04.454964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.497247 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:04.520571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74028ac-221e-4091-bad2-dc0217770e36 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.525556 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.526162 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.526441 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.526710 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.527346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.529310 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.643854 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1417/2859] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:37:04 2022] POST /identity/v3/projects => generated 373 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:37:04.648175 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.659061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e2c7d-54dd-45a0-ad90-6f35a3cd8c59 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.690046 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:04.719861 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.720628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.720628 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.720883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] PATH_INFO: `/v3/users` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.721537 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.723557 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.801734 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1443/2860] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:37:04 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:37:04.806417 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.817736 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7483a6-8b39-451c-824b-43d0d58b7d36 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.822585 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.823187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.823464 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.823731 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.824471 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.826086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.837178 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1418/2861] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 10 11:37:04.841815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.852488 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e511e4f-ba66-43a6-a2e2-13fc5e79872d admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:04.857132 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:04.857823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:04.858107 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:04.858390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] PATH_INFO: `/v3/projects/20a7bd7450a743e99f73fec74c4398b9/users/4988a6d6d0904b08ac941767246e5221/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:04.873928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a7bd7450a743e99f73fec74c4398b9, user_id=4988a6d6d0904b08ac941767246e5221, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:04.876600 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:04.945946 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1444/2862] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:37:04 2022] PUT /identity/v3/projects/20a7bd7450a743e99f73fec74c4398b9/users/4988a6d6d0904b08ac941767246e5221/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:04.950312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:04.961329 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5d872-5a4b-4235-892f-c5f28a330734 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:05.029939 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19e20c-6b27-46ce-9b3c-be6a54c2fce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:05.030618 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e20c-6b27-46ce-9b3c-be6a54c2fce8 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:05.030953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e20c-6b27-46ce-9b3c-be6a54c2fce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:05.031322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e20c-6b27-46ce-9b3c-be6a54c2fce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:05.032307 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19e20c-6b27-46ce-9b3c-be6a54c2fce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:05.034680 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19e20c-6b27-46ce-9b3c-be6a54c2fce8 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:05.047405 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1419/2863] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:37:05.051565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:05.061743 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b5753-1a5d-4447-836d-434312cca549 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:05.066835 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:05.067487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:05.067767 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:05.068221 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] PATH_INFO: `/v3/projects/20a7bd7450a743e99f73fec74c4398b9/users/4988a6d6d0904b08ac941767246e5221/roles/e3983951d6d24c7dadec4aa8c559b5ae` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:05.070309 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a7bd7450a743e99f73fec74c4398b9, user_id=4988a6d6d0904b08ac941767246e5221, role_id=e3983951d6d24c7dadec4aa8c559b5ae)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:05.073056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:05.084659 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1445/2864] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:37:05 2022] PUT /identity/v3/projects/20a7bd7450a743e99f73fec74c4398b9/users/4988a6d6d0904b08ac941767246e5221/roles/e3983951d6d24c7dadec4aa8c559b5ae => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:05.151094 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:05.151523 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:05.151985 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:05.180591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] MFA Rules not processed for user `4988a6d6d0904b08ac941767246e5221`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:05.181806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:05.255724 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1420/2865] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:05.267928 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:05.269648 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:05.331722 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e395be-2705-4d57-84dd-c9d2f0eda1d4 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:05.335420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127d7a66-a4c7-4edd-930c-403412637315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:05.336056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7a66-a4c7-4edd-930c-403412637315 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:05.336337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7a66-a4c7-4edd-930c-403412637315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:05.336609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127d7a66-a4c7-4edd-930c-403412637315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:05.394693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7a66-a4c7-4edd-930c-403412637315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:05.396194 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127d7a66-a4c7-4edd-930c-403412637315 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:05.404950 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1446/2866] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:08.496849 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:08.531073 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:08.558148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a9dce-7b46-416a-bbea-ac6fc2923d26 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:08.563311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:08.563938 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:08.564242 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:08.564516 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] PATH_INFO: `/v3/users/1307001c86e84466b939f872f9385516` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:08.565668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1307001c86e84466b939f872f9385516)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:08.567822 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:08.825148 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1307001c86e84466b939f872f9385516'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:08.825829 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1307001c86e84466b939f872f9385516'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:08.826430 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1307001c86e84466b939f872f9385516'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:08.834240 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1307001c86e84466b939f872f9385516'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:08.903578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1307001c86e84466b939f872f9385516'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:08.920941 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1421/2867] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:37:08 2022] DELETE /identity/v3/users/1307001c86e84466b939f872f9385516 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:08.925855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:08.925855 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:08.926441 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:08.945227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:08.946411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:08.979624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:09.005839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1447/2868] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:37:09.015328 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:09.017336 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:09.073896 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d3f73b-6e42-4377-b870-c2e9a2ddaa3a admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:09.077493 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:09.078024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:09.078337 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:09.079000 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:09.087199 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:09.088757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:09.098331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1422/2869] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:09.465485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:09.466938 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:09.482421 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78314157-774f-404c-93cd-995ad00abcd4 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:09.489353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:09.489989 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:09.490264 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:09.490531 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] PATH_INFO: `/v3/projects/a432c036587d4010a926f3b3cb86911d` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:09.492445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a432c036587d4010a926f3b3cb86911d)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:09.494381 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:09.616611 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a432c036587d4010a926f3b3cb86911d', 'request_id': 'req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.621967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a432c036587d4010a926f3b3cb86911d', 'request_id': 'req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.628210 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a432c036587d4010a926f3b3cb86911d', 'request_id': 'req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.718373 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] The token cache is being invalidate because project a432c036587d4010a926f3b3cb86911d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:37:09.718663 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.719795 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.721287 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1448/2870] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:37:09 2022] DELETE /identity/v3/projects/a432c036587d4010a926f3b3cb86911d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:09.725276 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:09.735683 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1917a3-1246-461c-bfef-762c7adaa9a6 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:09.784312 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:09.796354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:09.796354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:09.796354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:09.796354 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] PATH_INFO: `/v3/users/4988a6d6d0904b08ac941767246e5221` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:09.796752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4988a6d6d0904b08ac941767246e5221)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:09.796752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:09.978885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4988a6d6d0904b08ac941767246e5221'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.979498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4988a6d6d0904b08ac941767246e5221'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.980143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4988a6d6d0904b08ac941767246e5221'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:09.986888 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4988a6d6d0904b08ac941767246e5221'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.069359 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8cef0ab3-2c4b-4161-bdc7-9e9868d89589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4988a6d6d0904b08ac941767246e5221'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.087795 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1423/2871] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:37:09 2022] DELETE /identity/v3/users/4988a6d6d0904b08ac941767246e5221 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:10.512813 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:10.515784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:10.529072 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0f18b-6395-4d10-b4e1-3c3b0ad346d0 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:10.608766 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:10.609393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:10.610624 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:10.610934 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] PATH_INFO: `/v3/projects/20a7bd7450a743e99f73fec74c4398b9` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:10.612078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a7bd7450a743e99f73fec74c4398b9)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:10.613967 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:10.718710 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a7bd7450a743e99f73fec74c4398b9', 'request_id': 'req-c2324ef2-2d06-42d0-91a8-df740f65ed0b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.724088 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a7bd7450a743e99f73fec74c4398b9', 'request_id': 'req-c2324ef2-2d06-42d0-91a8-df740f65ed0b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.730218 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a7bd7450a743e99f73fec74c4398b9', 'request_id': 'req-c2324ef2-2d06-42d0-91a8-df740f65ed0b'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.731919 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:10.788382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] The token cache is being invalidate because project 20a7bd7450a743e99f73fec74c4398b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:37:10.788763 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.789911 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c2324ef2-2d06-42d0-91a8-df740f65ed0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:37:10.791464 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1449/2872] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:37:10 2022] DELETE /identity/v3/projects/20a7bd7450a743e99f73fec74c4398b9 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:37:10.796651 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:10.797761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:10.797761 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:10.929612 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:10.930825 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:10.988759 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1424/2873] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:10.996438 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:10.996718 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:10.996986 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.015883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:11.017202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:11.024870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1450/2874] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:11.029114 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.039155 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6b0a1-a679-4ccb-b207-0e215fe195da None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.043716 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.044346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.044620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.044891 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.045531 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 10 11:37:11.045833 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.048543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.059853 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1425/2875] 10.222.0.13 () {58 vars in 1153 bytes} [Wed Aug 10 11:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 10 11:37:11.064041 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.073320 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4ba0a-c662-4d68-8e79-424f7c00c7dd None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.077900 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.078601 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.079053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.079329 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] PATH_INFO: `/v3/projects` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.079983 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.081939 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.184793 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1451/2876] 10.222.0.13 () {60 vars in 1153 bytes} [Wed Aug 10 11:37:11 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:37:11.189058 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.199054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42238b1b-249b-4999-8d23-913641f61302 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.255363 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.255963 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.256272 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.256543 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] PATH_INFO: `/v3/users` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.257153 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.259087 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.268241 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:11.360861 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1426/2877] 10.222.0.13 () {60 vars in 1144 bytes} [Wed Aug 10 11:37:11 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 10 11:37:11.364912 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.375233 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9689b40-2caa-4e06-b8e8-a35f249b5ddf admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.379984 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.380627 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.380905 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.381172 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] PATH_INFO: `/v3/roles` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.381812 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.383485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.394177 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1452/2878] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:37:11.398327 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.409143 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd93f9b-4160-4339-bdf5-bb7d9bc63339 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.413974 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f889fc3-1660-4174-9680-dde1e85dd9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.414565 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f889fc3-1660-4174-9680-dde1e85dd9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.414844 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f889fc3-1660-4174-9680-dde1e85dd9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.415116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f889fc3-1660-4174-9680-dde1e85dd9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.415752 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f889fc3-1660-4174-9680-dde1e85dd9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.417370 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f889fc3-1660-4174-9680-dde1e85dd9a0 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.427827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1427/2879] 10.222.0.13 () {58 vars in 1122 bytes} [Wed Aug 10 11:37:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 10 11:37:11.431909 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.442012 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf933df-b33e-466c-8e68-acf137fa472f admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.446588 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.447227 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.447507 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.447780 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] PATH_INFO: `/v3/projects/a8deb71dfa2045f2819d87ee669fcc65/users/a551273dd0384df392f9c391357f7fbf/roles/33b4484deb1044fdb1ab7ae8ebc74f8f` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.462592 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8deb71dfa2045f2819d87ee669fcc65, user_id=a551273dd0384df392f9c391357f7fbf, role_id=33b4484deb1044fdb1ab7ae8ebc74f8f)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.465509 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.526565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1453/2880] 10.222.0.13 () {60 vars in 1483 bytes} [Wed Aug 10 11:37:11 2022] PUT /identity/v3/projects/a8deb71dfa2045f2819d87ee669fcc65/users/a551273dd0384df392f9c391357f7fbf/roles/33b4484deb1044fdb1ab7ae8ebc74f8f => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 10 11:37:11.532293 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fa892-a8c2-4047-a60c-74ef442b8c0d None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.532687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fa892-a8c2-4047-a60c-74ef442b8c0d None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.532969 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fa892-a8c2-4047-a60c-74ef442b8c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.551620 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-594fa892-a8c2-4047-a60c-74ef442b8c0d None None] MFA Rules not processed for user `0b3cf49f311d4c78b75ab2d7b48164db`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:11.552889 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-594fa892-a8c2-4047-a60c-74ef442b8c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:11.615498 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1428/2881] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 10 11:37:11.622286 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:11.624153 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:11.682535 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b1c3b-2674-4c3c-9c16-0a7c1a5e4188 admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:11.686198 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:11.688244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:11.688244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:11.688244 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:11.697193 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:11.698619 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:11.708251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1454/2882] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:12.924619 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d6dee-63e0-4fac-b419-12e3546570de None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:12.925466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d6dee-63e0-4fac-b419-12e3546570de None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:12.925466 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d6dee-63e0-4fac-b419-12e3546570de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:12.955459 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-fd0d6dee-63e0-4fac-b419-12e3546570de None None] MFA Rules not processed for user `a551273dd0384df392f9c391357f7fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:12.956727 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-fd0d6dee-63e0-4fac-b419-12e3546570de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:13.021821 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1429/2883] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:13.032430 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:13.034035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:13.091745 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76015411-17b6-4081-bb40-22a6eca748ab service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:13.095494 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:13.096124 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:13.096422 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:13.096687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:13.104821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:13.106220 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:13.115500 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1455/2884] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:15.867346 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:15.867885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:15.867885 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:15.889713 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] MFA Rules not processed for user `a551273dd0384df392f9c391357f7fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:15.891917 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:15.899846 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1430/2885] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:16.133035 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:16.134446 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:16.146463 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a553ef5c-ea04-4d3d-b5f4-6ef1ca3332c1 service cinder] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:16.149987 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:16.150570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:16.150844 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:16.151113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:16.152801 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:16.163382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:16.164817 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb39b8e-b2a4-4b85-8e92-0ae54c1e5cec service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:16.173353 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1456/2886] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:17.585064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:17.586454 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:17.659311 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58601bf5-30b0-4854-b62c-cdae10b95909 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:17.664424 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:17.665026 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:17.665303 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:17.665578 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:17.667289 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:17.675403 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:17.679150 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:17.680757 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:17.690986 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1431/2887] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:20.127315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:20.127846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:20.127846 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:20.148660 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] MFA Rules not processed for user `63e9c06c2c4f4caabf6ecd67cece94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:20.149874 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:20.163345 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:20.208652 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1457/2888] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:26.836858 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:26.847461 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a164820-e5bb-4191-8d76-653100b9d18c service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:26.851282 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:26.856074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:26.856074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:26.856074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:26.856074 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:26.939163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:26.940791 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:26.955118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1432/2889] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:28.157816 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:28.170820 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707272c1-5d4d-4aa6-b4f9-a165b8c4a622 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:28.176940 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:28.177558 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:28.177840 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:28.178185 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:28.180481 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:28.195061 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:28.196523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33d8fe-c7c2-4922-b4d6-03339c13a61f service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:28.208188 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1458/2890] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:28.691654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:28.693187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:28.707339 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbca19-09fe-486f-92b3-6186039b51f1 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:28.712913 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:28.713654 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:28.713942 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:28.714217 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:28.716206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:28.728322 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:28.729890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:28.740841 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1433/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:38.106389 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:38.106867 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:38.106969 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:38.126423 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] MFA Rules not processed for user `a551273dd0384df392f9c391357f7fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:37:38.127594 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:38.134206 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1459/2892] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:37:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:37:38.365142 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:38.374649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a1e5-24a2-469a-8a31-c249917aad1a service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:38.378914 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:38.379537 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:38.379818 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:38.380144 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:38.381912 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:38.391887 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:38.393331 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9c72e-9ed8-4324-a4a6-232d41d5c88a service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:38.401450 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1434/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:44.000631 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:37:44.010192 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9240376-0b73-4ae9-a749-fb330789a2e2 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:37:44.013686 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:37:44.014269 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:37:44.014541 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:37:44.014807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:37:44.017078 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:37:44.079456 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:37:44.080913 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:37:44.090442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1460/2894] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:37:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:37:48.397681 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=49 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:37:54.086486 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=55 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:38:05.578785 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f191a-842e-40d1-809d-ad3327ee9bd7 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:05.579273 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f191a-842e-40d1-809d-ad3327ee9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:05.579381 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f191a-842e-40d1-809d-ad3327ee9bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:05.598553 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-9a5f191a-842e-40d1-809d-ad3327ee9bd7 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:38:05.599771 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-9a5f191a-842e-40d1-809d-ad3327ee9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:05.656478 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1435/2895] 10.222.0.13 () {58 vars in 1009 bytes} [Wed Aug 10 11:38:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:38:05.793376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:38:05.794821 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:05.864599 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23667d2f-bc3b-43aa-aa0a-5fe8646f6a9b service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:38:05.869784 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:38:05.870560 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:05.870843 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:05.871119 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:05.879754 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:38:05.881357 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:38:05.889491 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1461/2896] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:38:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 10 11:38:15.648469 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=25 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:38:15.886171 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=32 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:38:22.641452 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] REQUEST_METHOD: `POST` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:22.641999 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:22.641999 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:22.660355 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.auth.core [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] MFA Rules not processed for user `832be98ac39748009caa1124bf1b3a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:38:22.661533 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:22.668455 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1436/2897] 10.222.0.13 () {58 vars in 967 bytes} [Wed Aug 10 11:38:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:38:22.681114 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:38:22.692851 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eae7dc-2853-4492-a64f-0545c2059e11 service swift] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:38:22.697823 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfb941d3781044b0a2092c596bf51e32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:38:22.698440 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:22.698716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:22.699121 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:22.707209 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:38:22.708814 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:38:22.717202 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1462/2898] 10.222.0.13 () {60 vars in 1384 bytes} [Wed Aug 10 11:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 10 11:38:25.654394 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:38:25.656477 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:25.671826 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08302e1-2aab-404a-9f47-bd632550ec66 None None] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:38:25.676872 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:38:25.677463 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:25.677738 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:25.678004 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:25.679813 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:25.692999 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:38:25.694591 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:38:25.705688 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1437/2899] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:38:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:38:25.810234 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:38:25.821790 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4385d8d6-158d-4399-a28b-8317e46271b3 service swift] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:38:25.828965 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:38:25.829612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:25.829890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:25.830190 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] PATH_INFO: `/v3/users/a551273dd0384df392f9c391357f7fbf` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:25.831318 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a551273dd0384df392f9c391357f7fbf)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:38:25.833290 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:38:26.006249 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a551273dd0384df392f9c391357f7fbf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.006869 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a551273dd0384df392f9c391357f7fbf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.007471 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a551273dd0384df392f9c391357f7fbf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.014390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a551273dd0384df392f9c391357f7fbf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.094775 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a551273dd0384df392f9c391357f7fbf'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.113057 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1463/2900] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:38:25 2022] DELETE /identity/v3/users/a551273dd0384df392f9c391357f7fbf => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:38:26.370933 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:38:26.382006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5930a05-51ab-46fe-acce-ea06bef2ff9f service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:38:26.443627 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:38:26.444308 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:26.444586 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:26.444860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] PATH_INFO: `/v3/projects/a8deb71dfa2045f2819d87ee669fcc65` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:26.445964 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8deb71dfa2045f2819d87ee669fcc65)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:38:26.447860 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:38:26.598075 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8deb71dfa2045f2819d87ee669fcc65', 'request_id': 'req-8476df94-e549-4dc0-b992-6ff09eb2b431'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.604050 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8deb71dfa2045f2819d87ee669fcc65', 'request_id': 'req-8476df94-e549-4dc0-b992-6ff09eb2b431'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.610530 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8deb71dfa2045f2819d87ee669fcc65', 'request_id': 'req-8476df94-e549-4dc0-b992-6ff09eb2b431'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.683248 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] The token cache is being invalidate because project a8deb71dfa2045f2819d87ee669fcc65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:38:26.683890 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.684086 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:38:26.685570 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1438/2901] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:38:26 2022] DELETE /identity/v3/projects/a8deb71dfa2045f2819d87ee669fcc65 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:38:36.109251 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:38:36.679936 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:38:48.485393 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:38:48.486833 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:48.543348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eb1ba5-755b-4b1f-9335-d13d3ba55ecc admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:38:48.591289 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:38:48.591803 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:38:48.592126 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:38:48.592420 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:38:48.594157 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:38:48.603815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:38:48.605274 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666939e0-0fa0-468a-8316-f23ac6ea1b2d service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:38:48.613740 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1464/2902] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:38:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:38:58.610156 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=46 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:40:07.394091 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:40:07.396737 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:40:07.499272 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476df94-e549-4dc0-b992-6ff09eb2b431 admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:40:07.499272 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:40:07.499676 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:40:07.499955 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:40:07.500367 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:40:07.502398 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:40:07.517038 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:40:07.518907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:40:07.531506 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1439/2903] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:40:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:40:09.177815 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:40:09.178337 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:40:09.178453 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:40:09.197839 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] MFA Rules not processed for user `63e9c06c2c4f4caabf6ecd67cece94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:40:09.206565 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:40:09.254063 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1465/2904] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:40:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:40:17.525763 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=34 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:40:19.246853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=25 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:40:29.161907 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:40:29.167649 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c24c4-b4cc-4370-94d8-12e45e442a2b service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:40:29.173850 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:40:29.174444 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:40:29.174721 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:40:29.174988 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:40:29.176781 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:40:29.189668 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:40:29.196390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:40:29.202022 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1440/2905] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:40:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:40:39.197292 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:41:54.778382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:41:54.779807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:41:54.920632 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7587357-8e17-4057-a93e-40b4cd08c137 None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:41:54.925918 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:41:54.926523 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:41:54.926837 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:41:54.927104 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:41:54.928853 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:41:54.940736 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:41:54.943721 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacbd8d-0f27-40dd-91d1-fe3da318327c service placement] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:41:54.952223 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1466/2906] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:41:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:04.947978 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=63 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:42:17.947807 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:17.949219 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:18.053579 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d91c1-71c7-4d00-a724-f549c70fa038 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:18.057877 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:18.058480 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:18.058756 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:18.059024 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:18.060739 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:18.071823 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:18.073278 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:18.082005 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1441/2907] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:28.077976 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=48 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:42:30.729545 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] REQUEST_METHOD: `POST` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:30.730158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:30.730158 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:30.751467 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.auth.core [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] MFA Rules not processed for user `63e9c06c2c4f4caabf6ecd67cece94f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 10 11:42:30.752692 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:30.759459 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1467/2908] 10.222.0.13 () {58 vars in 958 bytes} [Wed Aug 10 11:42:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 10 11:42:30.797692 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:30.799169 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:30.810204 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c7bd47-4a67-4954-9de2-bf334bb6a8a6 service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:30.813789 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3cada6c06ef8427395eda888397efac4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:30.814377 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:30.814655 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:30.814928 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:30.816594 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:30.850115 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:30.851527 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:30.872790 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1442/2909] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:31.155894 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:31.157323 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:31.191016 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0edfb0e-3c76-4656-87a4-47a1a23e92ef None None] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:31.194577 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:31.195169 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:31.195445 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:31.195714 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:31.197427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:31.206644 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:31.208094 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:31.215872 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1468/2910] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:33.033006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:33.043007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22f07c-c2ce-40a2-907b-42dea73877cf service cinder] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:33.046571 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:33.047194 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:33.047500 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:33.047773 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:33.049574 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:33.062916 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:33.064393 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:33.075306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1443/2911] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:34.032288 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:34.033690 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:34.049029 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbbaa1-cb74-46be-8966-f57f13b1dc0f service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:34.053996 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:34.054612 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:34.054892 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:34.055162 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:34.056865 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:34.069322 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:34.070885 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:34.081056 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1469/2912] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:43.070298 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=24 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:42:44.076771 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=24 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:42:53.525118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:53.526706 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:53.540373 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292a4e6-4d15-4147-ac92-7f6b3ffb6d50 service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:53.546400 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '462638e8386c49668290954c0211728c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:53.547006 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:53.549769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:53.549769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:53.549769 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:53.561685 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:53.563245 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:53.573806 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1444/2913] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:42:57.489693 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:42:57.499620 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2e418-4c4f-4f95-b7ef-aba50108366c service placement] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:42:57.503485 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:42:57.504113 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:42:57.504390 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:42:57.504657 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:42:57.506376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:42:57.569402 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:42:57.570807 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:42:57.581164 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1470/2914] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:42:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:43:03.569255 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:43:07.576971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=30 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:43:52.500881 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:43:52.502252 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:43:52.514520 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d611477-3120-49f5-b9f5-08ed04347982 service nova] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:43:52.518083 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:43:52.518671 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:43:52.518953 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:43:52.519225 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:43:52.526390 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:43:52.527793 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:43:52.535646 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1445/2915] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:43:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:44:02.532127 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=6 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:45:07.139870 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:45:07.149411 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46cc78-df52-49a3-a4ee-56f6ceffc0f4 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:45:07.153013 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52773c21d6e9406086113a9fd74d5f62', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:45:07.153609 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:45:07.153883 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:45:07.154149 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:45:07.155828 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:45:07.237806 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:45:07.239256 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:45:07.257891 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1471/2916] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:45:10.532690 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:45:10.545575 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb13e0-d57f-4399-be99-ed29c761b94c service neutron] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:45:10.551054 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:45:10.552923 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] REQUEST_METHOD: `GET` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:45:10.553318 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:45:10.553687 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:45:10.555809 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:45:10.569323 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:45:10.571070 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:45:10.581824 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1446/2917] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:45:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:45:14.237707 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:45:14.240319 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:45:14.252674 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0761ef-ab79-442f-a876-efde9311d201 service neutron] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:45:14.257931 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:45:14.258605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:45:14.258605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:45:14.258605 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] PATH_INFO: `/v3/users/63e9c06c2c4f4caabf6ecd67cece94f4` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:45:14.260376 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e9c06c2c4f4caabf6ecd67cece94f4)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:45:14.262687 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:45:14.423561 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e9c06c2c4f4caabf6ecd67cece94f4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.424271 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e9c06c2c4f4caabf6ecd67cece94f4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.424879 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e9c06c2c4f4caabf6ecd67cece94f4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.432388 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e9c06c2c4f4caabf6ecd67cece94f4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.488461 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e9c06c2c4f4caabf6ecd67cece94f4'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.506382 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1472/2918] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:45:14 2022] DELETE /identity/v3/users/63e9c06c2c4f4caabf6ecd67cece94f4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:45:14.741732 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:45:14.752306 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f6d35-b555-494e-bf1f-f6bf9a952cf8 service placement] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:45:14.809779 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:45:14.810406 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:45:14.810714 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:45:14.810983 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] PATH_INFO: `/v3/projects/e8775e002af14ee2ac9d6601a6872dfa` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:45:14.812116 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8775e002af14ee2ac9d6601a6872dfa)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:45:14.814101 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:45:14.946847 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8775e002af14ee2ac9d6601a6872dfa', 'request_id': 'req-6e628da7-cc03-4c87-a400-070d6883fdde'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.953163 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8775e002af14ee2ac9d6601a6872dfa', 'request_id': 'req-6e628da7-cc03-4c87-a400-070d6883fdde'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:14.961827 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8775e002af14ee2ac9d6601a6872dfa', 'request_id': 'req-6e628da7-cc03-4c87-a400-070d6883fdde'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.025873 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] The token cache is being invalidate because project e8775e002af14ee2ac9d6601a6872dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:45:15.026187 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.026820 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.028358 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1447/2919] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:45:14 2022] DELETE /identity/v3/projects/e8775e002af14ee2ac9d6601a6872dfa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:45:15.032487 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:45:15.033914 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:45:15.046955 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00510d0-893d-4eaa-987a-3f4da4bef46c admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:45:15.079651 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:45:15.103037 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:45:15.103716 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:45:15.104020 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:45:15.104315 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] PATH_INFO: `/v3/users/b336e6b2105e48df8c6fa98baf485c20` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:45:15.105442 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b336e6b2105e48df8c6fa98baf485c20)` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:45:15.107348 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:45:15.290241 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b336e6b2105e48df8c6fa98baf485c20'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.290921 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b336e6b2105e48df8c6fa98baf485c20'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.291386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b336e6b2105e48df8c6fa98baf485c20'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.298305 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b336e6b2105e48df8c6fa98baf485c20'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.356890 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.notifications [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b336e6b2105e48df8c6fa98baf485c20'} {{(pid=80629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.379386 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1473/2920] 10.222.0.13 () {58 vars in 1224 bytes} [Wed Aug 10 11:45:15 2022] DELETE /identity/v3/users/b336e6b2105e48df8c6fa98baf485c20 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:45:15.599051 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Authenticating user token {{(pid=80628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:45:15.601064 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.fernet_utils [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:45:15.618251 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e628da7-cc03-4c87-a400-070d6883fdde admin admin] Validating token access rules against request {{(pid=80628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:45:15.675991 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3cf49f311d4c78b75ab2d7b48164db', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1f5409bf396e4a7cbe05519266660a63', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:45:15.676650 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:45:15.676932 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:45:15.677206 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] PATH_INFO: `/v3/projects/b5c98518cadb400a8fe2ce233d32257a` {{(pid=80628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:45:15.679665 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c98518cadb400a8fe2ce233d32257a)` {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:45:15.681632 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] RBAC: Authorization granted {{(pid=80628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:45:15.702389 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:45:15.769340 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c98518cadb400a8fe2ce233d32257a', 'request_id': 'req-4dfb7a02-6769-473f-bded-9ee1ffd895eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.775564 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c98518cadb400a8fe2ce233d32257a', 'request_id': 'req-4dfb7a02-6769-473f-bded-9ee1ffd895eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.781007 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c98518cadb400a8fe2ce233d32257a', 'request_id': 'req-4dfb7a02-6769-473f-bded-9ee1ffd895eb'} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.865965 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] The token cache is being invalidate because project b5c98518cadb400a8fe2ce233d32257a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 10 11:45:15.866190 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.866816 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG keystone.notifications [None req-4dfb7a02-6769-473f-bded-9ee1ffd895eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 10 11:45:15.868342 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: [pid: 80628|app: 0|req: 1448/2921] 10.222.0.13 () {58 vars in 1233 bytes} [Wed Aug 10 11:45:15 2022] DELETE /identity/v3/projects/b5c98518cadb400a8fe2ce233d32257a => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 10 11:45:25.371427 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:45:25.886118 dv-cinder-850618-2-iscsi devstack@keystone.service[80628]: DEBUG dbcounter [-] [80628] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 10 11:46:56.744944 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Authenticating user token {{(pid=80629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 10 11:46:56.746414 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:46:56.817571 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799bcd3-9f47-4dbd-b553-038a6d12c16e admin admin] Validating token access rules against request {{(pid=80629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 10 11:46:56.875099 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8593dfcc1d0542bfbaa6fb169b01ab90', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ce1741f207fa4b4fb6798f37092a4a10', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 10 11:46:56.875697 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] REQUEST_METHOD: `GET` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 10 11:46:56.875971 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 10 11:46:56.876265 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 10 11:46:56.878053 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.fernet_utils [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 10 11:46:56.889018 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 10 11:46:56.890570 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef2eeae-1fc1-46b0-9aa8-f652227460e3 service placement] RBAC: Authorization granted {{(pid=80629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 10 11:46:56.900786 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: [pid: 80629|app: 0|req: 1474/2922] 10.222.0.13 () {60 vars in 1365 bytes} [Wed Aug 10 11:46:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 10 11:47:06.896586 dv-cinder-850618-2-iscsi devstack@keystone.service[80629]: DEBUG dbcounter [-] [80629] Writing DB stats keystone:SELECT=58 {{(pid=80629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}